TTIC 31190: Natural Language Processing

Transkrypt

1 TTIC 31190: Natural Language Processing Kevin Gimpel Spring 2018 Lecture 17: Machine TranslaDon; SemanDcs

2 Roadmap words, morphology, lexical semandcs text classificadon simple neural methods for NLP language modeling and word embeddings recurrent/recursive/convoludonal networks in NLP sequence labeling, HMMs, dynamic programming syntax and syntacdc parsing machine transladon semandcs 2

3 3

4 Google s NMT System 4

5 Google s NMT System 5

6 Google s NMT System they use a procedure that determinis1cally segments any character sequence into wordpieces vocab: 8k- 32k wordpieces they first learn a wordpiece model 6

7 Google s NMT System 7

8 8

9 9

10 Noisy Channel Model from Noah Smith

11 <latexit sha1_base64="pw8oji3ujegpi9vjlgbxuppo7ly=">aaa0ohicnvvrct22ezbtw6rekuzn2xk2kqzylkk6jpnmkvfmffu1o3euxzzzsshrqbkhbxjvbsbzmcp+7tp2ifoo3qv4dkkaloroxhybfltylhyxi4ucimfc7u//99zbp/v5l375q7d/vf6b3/7u /fivykof0ezgnof8+iiimlkppjzmj/b7glkrbqr8llh9g+3dtygxlsxo5kohzsukmjvlijfsdv/ofrswprp986mu5e4twqrl1rdfn7gw1/ep27d33yj+hy7kd6krdevte8xxrpufvxf1r/blye+gsiucvnna19x3us3qffx1+sfz2oysn8vb68by/pagsfvxtdt/lszinkf8/f2djf29f/xj2x6j52fhrfo7p3/3wlh/lyznstiyjeej0tf/is4pwycke1ut+kwhbwkss01p4zehkxvmlnfz7w1ateeocw79meqq2s1grvihfggayjxiizdasdlwdlnl8yvnfsqkunat1r+my8wtuofq9iheyflkadxjybrj6ohroqgmttn7rpiajtfkw94zsstqxyz4w/dqyk2lcwnm/ni0tyxg+69eg+zrlscyzfvu4bb5ljvu1foahcikpwrkgzsvyglxrde0wru0xsvp2mn6w46044lcm97wzcdrxjmrkweg8acqgjhdqoam0u4r9dac5p2dcqw1ogpyk/tprbmmwm6pfce9v8vyiyvvau/qgoa2rrmbhszzd/ezsu8bqa6ptgrelmbucdmg4mb5mbryhrapbdhkkn5mhkvdhp6suk5wvkoe7hoyixmuidjipkqkgwctqcj5reza8imu/m1n5yk0jmhsgflcvkiuevlkox7s4x33m1dcw0rnnpibkhg5vndmnxpvmtorkd2jmskpnlnp8apegevaapgkfc5puvpo6wttvpk3rbomimleifkba3/vibiuakxiohdq9qe+isobdkoyqhgkmwutwr6txghekbor8yylgbxhpfto6q3ztojanvzcufec19rryvfnvy+tt7n55/ois0caks0beg3i5onh2zgldoaxjd55u6jhiomz65ugr0dnrcajepl4afvfepty8glwqhp9f0ypn0riudo96qk8eweinmubqwo9fhnvtkhawgljjhndhc+c46t97vckgn00afqu+inao1hn4au/fpqfjamnkssqlgjcupjulwa98gcuxwk2erhc8p+lttoxncwu0tzxpmicjbdbytvutsz63lgvhlmspoelneyf3fbv8couclbflbei7mvlombkpy5tsv5ptuslsmk/zicz9ugmbkade9y7+bab1wfhhvynqwpmkwsuwt9abgosylok4iilfrt9fvndccnbcb7jrqikawbkmulxwcfdndysxdvkw5dhittcllsrbcla8ii5zxjdzed6y56l3dhdw4ilxkilyt201gynbymimwqc4vmdierocwzmhopjinfmtlkxgckfdfeev3/ddxng/evpba1bkziehf2k36ygsbboinmewqeqenjlfeijhncunana7s/rgjvz6vgky6wxwhezfhen2b7pf7e5qqrejdgkcworbwtwb7fyppj4vohm8pd9z3lbb2ilp1r4gtmtbmqpnxbsvkoduldlumt073ki5ven14ek7dvi2wnvnt5st9hcyi+sfb2xwru0r8d7mxmhzwlqpvzp6crw07tdkz85jjhobmax00z63tmxauelntkjrqoojd0kqtj10z2dikbv8sgaaijqknlqqkjuqkyfciqqtbipmjoh3jdml9cu7buhv3clg22hn0o5olgsxpjzlpiichnnmzj+wkjyeud9auwzdnnbi0nakgfymlfrxvodydsls+znh99mw4bbszliq47cjllvj6ymykkywlo7zna8b8ozfkteynoimtkzotx+vkn2u8pimsyyemo5vkjvfhqetdkd7i5pasnhuvub/cyownqknnlyedpry05ygrphf/se4kun/zbmwlcjamcszegbchh3ydgx70qqsfayfxrqk4g1gnjmbggktnkr5ytv84wqgxlnjxhvgz1s2zfz4fnqnvfo04ccnvb96ih3mlluzqmr1nho221g52gdew9qmopn+bxplbcmrkrl1ygmxribqdvmuuyglyymn2rf8s9+9fqzxoczerzk2cklw7dqksfw9c6mwxqe71vptjvlifdamw7ingibw+f8gc9jfnf13ac3pkos2k6bksurnvlxqffhk8dn6v6nmvtrdtgpu/3adootp1jufgs6m1sm2lgrhrs+xzcbeouuankax6dy5z9ijzw7sfcwvbikuxbrewintcym1z5omu8cbfvad5bbokpjxk5v0ic+1twxxztzbnssxorgzhw40wexylvsuzqzsyy8dkkycl9xl0hr3vboro9mbf9c5ibinoedixg7ydn2h/chtga3qdsafamvfw/3f1vewejarta4wmscbvvhymgcnyglktusaks2xzgzrgzkbo2bkk7bpzi9ti+tjtsn7pgbktrs4bn4y6ido8larugz/muynbviwqw6q68sxgqo9qepwo/cipjj/gxltdllb3/xe20fe6tlhpv1/ayq2areeamagldocxpf2rk/afw7whtkmpgvohffkrvs7lxijstpncifajcozilxwn2no1m7pxk9kanfjjbsn2frdcpqsesdqurqvtl/l4tbeyx7lph0f5qkialjhmrkpmco9503auexhuqd18vgsgcz118unwph69o6zkcaoxi+rjbuke0wzylm4zisctsyxnozo6/4yjctjva50lpuyfjcbwzws7desbsbqmllhipuyfmomsk8mgl6vvalzpilvhzkbedg9dycv1egfjffexf0iohimfglygkj0j8jgilsgmajkpawuxftxuylx4pmc0qzqo6/rvkdxt47w0crs0nvujzlkfuoywfyljolqkrypkq4ttvgsnjxktpw01fqytmkcr2ra3kncynvnpi+gok4n1kvna8bib8bjpop6ww4ba6pz0qhleua7phv2zurfodymlcaumwwj2ruke+vk0hpsbnpcdkgdqprpmnvqqxkehkbuczzwj4w95jlqntrgeabpdmw+i0k1w11zixayrms34lrfkmydb1ze0yac68aguzxg7usq0u8nnmsifjzhlwb0qvd9q1zz8+ngskfiz+zkmzs9xvqvreouzu0afsxlknfwwiyyfxikanuxyo6wewdhqmgeqmoezntdire2opgfh8enxkmgjeoggvgnxjto0wsjh8y5vwazw6ay4ztcz0rl6lkwivzzuqpwksgguguzt/z1rztkbskst6ku4py4zgyz1ckp9mbowsfjkhrz1wmr/jzksedyt5vcl5lo5czpauwc8otmp1xpdm7xbrdbhbzraoyjm4islmlh/m6oaslb3swz6ckknusfsgy3xboko7yjpirg9xft+1zpgrl2xsobso/mm+xuemlov91br/n4ga8msbna1vr7ssbuuhg89jexfydq1rrxklqqsy6wmmq+xhh8ufjtwav7qjqjz6yigxyh04escqeceddupouixm4ulrnanez6ppjctter2ucf40aobbeouqvtm63z14fd4pvqdr5hou87sedivpfqbw+w4rvlpghi1ovam2jrvcmfxhrx6tbbhisjbg51i7aeycfellk8rcc33i/zf3zdv9gtrafegvg0eywnykg54dy1qsf+3x73zfzrv906+y91og1uz+lcvhur8pgkbflr5zzwy9tbilyvj1qyaeqvpoyzrhldsnnzdhat98x3obusgj2nz2ycehkzciguj7omvy9bbxlo0mz5e/0ger+2cwcdlwwgh6x53hbwd68drrva54lxnemxma1eegoe9ioryyndgbexusrkoz6hyfv5tws1ti1vmx0civtkeub+xxu4d2loa0mlhajmf2yucjvmacvlqwjkud83ewlvghkodzj0xqmowx9c5ov+j6y8eppbzphjvdriimqehcxp2lqgkllpljknyln1zmgsfnqoi1zdnc/wdd4zbjq7ethxs+bcgpaa6wo2zqhu+nrsbrzyl8chfizrjvkikrqv93eppuxojaaynpoudr9pvj+w79v6gpk1n2kla65jxts7h0sc8zpvrk50tsx+fmbda71hmtd1wyrspxkg4gyv/xslbxtxvna40c8yivs34jvojiog1myhqxnsvpgm8hlyzv4mvd2w9chskguw9vpjlthxr8broo/3pgrt2oofle9wxkdhommwvm8j0arppvxcvdz1p/1utzlvm2hkfrz2tamaxcebc5ukvdr9knt36qefebv/e558bi2nv8uevtxfqu0zukwnnavuh9vhgyzi9g/cuzov5c3pzhzkfri7b19rk4gu8vbwkxodoptmyvhzzg/36oz0mjjyoedqxk/rc+vrz9qvrumxdmza4nhowc6ghhtjtzqxsoxjia7ivazqma+q1vf1iq63x7b7oxj66lda9dudmo5h7lqufv+vjbwbezqharvq0h8/supnemuh10t5jzmpjesw0upy9axfex+czyubftuyqqgf0hba6fcwhv8x6fg57olcsnqdi2wlsonytrllfqtkg9gvksuhjmqtffmh3eqpiwmapculeuoq/blu+ox9efolbde1yld9tjlxp4x1xemaxhmhxhdnv+5d0ijm6igywuwd5mmle13gekiwkl0pd5p/ospckilgmlbn77o6at5obvxvam56silnrs7uklafnpvn1xgntgcpc56dtfnfb/xgvdedjhsvnvrye1xuqtaj55cdzfnd0qo1a24h+lctxhfn2wcyzmpii+qq+x0vdinh8xv/agzk/omm/fht3b3r/t7om3sbn3/s/pnm22t/wnt/bxtttpaptc/xhq8drz1fc9f+d+u9w3++9zd77997fo/re99o6fu3gpr31no/9178h/jgm6y=</latexit> <latexit sha1_base64="hzphb84eohhsi2oxga9qnsqx7tu=">aaa0t3icnvt7c9y2ezf7tnvx0v7zf9hkmtqxpoocp03b8uwu27u7cvtvj7xewqosob4kkqab8hrnhv0o/v79nt0feecsacu5mrffal9dlba7i8vduzkxqq4o/nfrbz/80y9/8tp3frb581/88le/fv+d33wpesvi+irmgrdfr0tsjbx0lwiqo1+xgpi8yuhx0cvdbp9qtovkvhipliu9zulascmliykqs/f/u1nk5estvwufz3izz0ixufp2jl2ub7+50wqpgjtm6ftdjkzv3gqq7wahrpkz+vzbphldhxl0lpczghqbjvgk6lmzxr49o199h4klm3vnc+f4djinsaoxtzizjdcfzxnnc/w1gra8qpkiis2z97co9g/0t+b+tnqpry325/jsg49uhqmpq5wwks6ilcetg1kd1kqofmcuefeslis+ick9gc+c5fse1lqrtbadnukw5ql+fsrqtx2kmurslvmikdlrm2m3yawv7ars009oa1aulajfbdqavlmgeiczeirmwnizjxyqwdcqnyc5ecrwmhebg25kkshcfxwwlfrrhyp5xg5ru0hkgysxw9q8yhqt/hjyg/m5k1lfi2h1nae+vaggtrtmhwqiqmuzbi7kvfq1ysvamow6yirm7c39cddyc8bvfe8hlydoljiroqvbdcqqgzhbrnulpyumhko754kczuenpqlqlg/rzbvnwwqpfid8ucw4qsuogsfownjyi03ajrkuw/zm8nvw0cor00sil2aqooiycta9mnoijyisrhwqym4yq5uywk5ipwzclckruwlmcmqriiqyrzkypexc0aheuhwypstlpwpvh2ctgtkuoh5qr1tljnyfhy9apqj/hjtoipssfnmqpuchqtvpk28qjmdnhuy0zuwtjxcdqqrhcbfqakyzbxoa1ageolo29qhnm36djrhkcfhmfbd/sd0kcscnajpobtcqjini4yjsqoecjnli1mzqcyb35atqwosyxw8yt3myoa1d4zowdxwuvrrhnq2mcoktygss7o19fvzktnxgmktb5dtyotp8ceoiqwqfyq+fb5vwsatm+vbji6uztzneffv0alrcefxo8dwosvf46poeazgntf34aid64okjfpzibg3m+gvcpyd5cqulm04ifyidx+3wxzskbr1ta+gb8fg0q63ox2/2/xmkoq23jg6rqiwje3q6ixt8klspeo3vwshmnwgtme9vbkhqqftb/llwmsigi1vx1tebcstzexopbgw1v5bitavz4jdffkrmwo4s3zwud8fjsjw6zhugbm9plpnfzhoadelmbeicede75hmyncnjhvulqqpnwyklic/qg6km2c6cudkrzgm+x9d3fqjqxa+yz0mswscyprl86+ggm1sw33ri40rgipxclne6bmpzcri6zqxdheggu8hz4ojw8ogzocqlfx9z1qymbculnwit+tibgys8brzqaxhw2zpmtfqrgctetv2e1whl9/yy+c5y3bmbiimhdgkvu9omksyzq2qixkmvikcriouqdonco0wl3l3rtzyrnmc1wwswcczk/i4ciruyrbz3c1unpxfifmfs383is8dudmdtxrkap7cxu8s7bhtbmtm6fmsllaxt4aypdnxlmkplnnumzzdyaoe3qr4+68yb307ropmzttubalnks+cads6mpsi+2n6abi+y9umvjp4dr0z7hdnzc1livkcmaln0562qlx0oowsvpowg5ux4iujatbq3m7ksoj5ysabekrlpogiubcicqc4r0w4dftzgdjtsgtlm9t0l9cbtepbttwz5tyeotusrx3krqa6oasbmt1iukwkoh6goijeaalfpbavajwywnxhqcrkwkzj8ybl7eqfwcdzlqqpp3foxrnh0qswuvjgw92loy4c8yglolayvtvmvjgzbjyribtd5syelgwizxzqvneoog9z7k/0jzr0kbldqq/8llzjoxdmax0agerftt1cm+wl/u9ziot/yjcwvdbymreaegbbhh64dw140gwqfq6g1lul4g5fnzsbgogunbj5fn5/7mifxbrnxjthzlguzfz6n9io1fi34yxy+dd2qpgvkuhnu5ey2gjbb1bny34owqr1ez8mozo4juqj0zbvapj5ya9wbtly2ua7ceru17/jnshsndojabyhzc1zgsq3yauyy+msfri+6hnbrtdpo8peoyhi2cbs1qqzz/1ewsl9pu+7rotjryg3xqmwyhnp4z6h77nnjv+hfrzp10xyxg9h/uw36lkzd1zqeurbwj9sykkwzvqsk2rdvfafwhszkcuchsy+u+bfxmd8fptj3urdpde4njm4+tzfbaw9b2deow3pb2l+dvnkhpdm+mcavuwzzgf/mycw8ojbgoencdsnswdnexkru2gsv64uvke7pzl93tgmur3najbpzwownhbdtukf7jd8bw6drgn80zoqh+7+roywghcobxwfkglet4qljdl6q18i8ybejw2gzy9yxmj92xmjwynvinrpgnstarvz0xmhcd/em/owoh3rb467og/zlcw/0w8cdclrdlz0rqn4d5aq3ck9ijvmfltelkbsmze+tg+s1njm20pxntmhrebugrgbsoe1hfo7o+hp87ggf22pcs4becqx3pd+dlcui02ngpewwumqheok/0zcpndej31qe4syunxqwqboqnfv3gnvn/ubrfyfabujaalye5poqkbaaci9lzr+xovdctgnntodnajtg0xudxc3x+q1yiebk3qmvwgjgr+ute5p7sgsqwlxihjxtjimejc6a4rgujmlujxsszbiwnxvdnazcmsxhxtaycsegzbiwegywryasxq+8evokkju9kpsqmbn1gk/16cvn8urc3y2ieiwweticqkwn0mkitjyxomqdpa7fw3o5ixhg+ylwro5jqnfor1dpjthrzfhsdvstktqvsrjyxkuk5vgsuqmsrho1vzkylerplyzvddk2cozhasbesczsfwdkjqb6tif1pwdnwhixwbm/hppl7gajaljrbeu8oh3sffcyyqgcz5nkyjdnxotsc4w99wuzcgkdxsf7pb4h7xtnoaghgujiw/clhvin0l3zpkgggjaigdr7iu+hun1w11yipybr2y281rdkv+erk1e0omg5mmh0pxu7krv0cwvniotf7velgf0u1f9sdnz8sjuxkfkpvvjm1wax1uxnukt3o4afe5kuvbuwc/yfbim7dizyo+wbqhoqaobqbghz9ojirs1ofbngcwpqekjreois6foubbrpgtfdcrguwmwyqmi6jqilyjg6im6jvvgtg1viiafvwewtf3prppobubjzaoo4j76zwuj3coi92fnwkhjl/vz12ik/tzjphevpku9s0pvc8jymxappbn7klxnwi3e4derhfudjz/xelbqzh/f7o4qi6hqzz6dljpusldkx3q7rk+zyjzjrgdfezd1zpq9kunsnd8o9mm+xu+mzo1t37xzn42e8mctvelsz7ckdu+lj8njhxgwbulvnxqvsqdy5wsqy+vhh8nnyvqmv3qnpgt6zjgbghwwfyqy9cezat5ooi5z6uxrmaneygflgxhxredlahedgqooqhlel7zqd89dh/ed7ya2+r1zkvvhw5kplmrchq/ckjvseetmiwjfe1xqpc4jp49jjgggfewjc+kzgbckcu1guvepat9kvurd2y1frs2wpx4ketjcvomiopehcc0lbwdaed+z2ybddoflptabtb2ds7t6klfh4rbwp9wpluduzbbc7lxm6mbt1snmmwng+idm+aae4jxvmu4jklscem7azaqwjtdxgikdds3raole59gkl3ka/kd6v7dzdwcedxudrfjce9d1rj1gj17loes14bgzrkd6bp/ti5glomeyktjyzwernyvh9vq9yrwzjs+yehyk9tp5f+pswhlzy00dymqmamx/ba9y2pudfcipbeft3zytaoyau8eggywv0iet3q3wx/m6wb29+uwxzapzatitso5afjtvx0cunjswlthuqytms1y8uyms6q/yznslnpm1nnwjnfsl8wbdsyh3bpiqbgp9a6oad+wx4mi9lgynlfewjcv4uslzkqmgkgejp0ver1vsj975tcfhccyo2voy65q2783eksk/zfvpez8fuz2nxvtubjntdl0xqddphic3w2s+0uez3zqunq/mie7j1k77jzoho4mx2vj7b/hvjhfeql4sqo/qbzqirliyxaj5k9sghfhyoev15/+m4bwl42qkel0roggcyxhqsiey1/yef69e6/qef+kmmfizmhcbcrmnsgdgkynfbsd/4x4xuxf30e2/jv8etz53v9bqzslf2w7e9alcmtloa6na7ffgkgv5rbnqmct6eztzoyrj3991rbzkqeh+xj025crxm82stowqwe/vdehhmetwgablx9yxy6gfqgtfzga82yumuc7azkofeuq/mfwz5bigdmlwrxsxrpa1ds4ib7rfsxg6fp3lcb4egarj3ohcjypqvzhlegqldwlrkkjmp7ywlib2yg+ulvjljhbii2ghlxd9r0d6f4dnkdti1bgvcrzsgfvoftog/snqsdnu6mkxf5yhv69iovpdiospq0ij2s+ziesnpb19vgdd5ttkujs9xsyqsbh+gg371yij+ccfnrxjuvg3m+psxquuagwvuguvuhjrdsj6kqj9e1pgdwpzzwsqxkqsqfwt/o932n4myeioguoz1+7upx7yfj5lqdzsna1gpemmlxtkak7bytd7u1q7gmrccrf0se+ch67obnkn47dtbnunyaijjuljoyz6biggow3n7wedd5azv3j7ecj5chthu7e+rybgm29/4rzmt+09k3i8v7+1pdvyn/76/9ekn7z/pvlfxu40/bnzemgz8zeptjacbxxuvnujbp7r14a2pbt2//9f7z/en92cg+onblc1vnwy/99/8h6itpg8=</latexit> <latexit sha1_base64="3ahrn4ufn5n9i409tvza2imd2fq=">aaa1l3icnvtbc922ezbdw6rekqyvnb6wldtksatqohwascbtklzrd+koii3nylhsgcqodytedidnyob9n33vd+kuwhniaqakrzo2codbxwkxu1hcfbqje3j//7+3bv/oxz/56c/e+fn6l375q1//5t33fvu1yese0pdhnut824aimrcmvprmjvtbglosbgn9jrh8io3ftckxlm+o5akgpymjmzzmizfqdf7erxors4rrx594wc7eipyqbh3rztk7q7zf36m9b17sj3qstwndttucyruel8r0vlp4mkrpkj+b3ijyxkfjxxvfqd2rv9g35xflb5+zeclvrg8dbfttgspqxvspizxdwntqsqxfx9ad+uri788ppj/wocxz82rzuntlnova6/cz13f8lm/knkdc99fdpp6yk7bq5fd6gnry36k7mbjdtaxvxhtf5vzdjf29ffxj2r+j5mnjrfk5on/vw1t+lidlsjmzjksik9f+iu8rwiulewqakwutshhjynocnxljqtitlkhu3hburn445/avk56q7vjujbvikqaatimcclmnk11tj6ucf3xasawojc1c3dg4tdyze2h1xsq4dd1zwacjoqnzpta1maqjtrne66ygcdhmlvegukk6l2gefv3amjniwsj5vzyte8l4puvxhvmuzbhms371oaw+zsb7trsmh3iiqcezbs7fwbi10qw9fubtmult9oz+sootooc3dpe8gqg68szksshvpaheii4xudmjnfoefxwnoafguufttokcpp06uqzjfpujxqnvvffcoml7wbmmheibi43ajpi4h/mzppemoqdgpzmifmaqooiqcta9mnosdoguthwysg4yqyxvw05iksc5x1dcdzjmeyixerkwuyvfqboiorg8opigeuikf2ay8sfae2qoqp2gxiexca2yhl5o8ad61kuhlamzmk1blawdqmqmt7wumzjulr/qmgwvmlymcwgewuraazgmfmxpuvlq6mhrv/s0rbsnimdgivczan/vvtkkytwf+mgyugf1srtjfbifxdvqgdfryep1nq7wjpranx9zxpgn46lprqevr10hpqekkpliqmaevo534m2mvn3dl46endbawhhjihhllochl04tyqinipnb800d/qmhwd98fgh09macqfdpr0zfffgphl+nklwpl6/pkwzpxfepd3uol59y4icxrtbaj1+e1xosfrbwwuoe0mdz4ljpv3hdwqa3tqb9dt6kdqju+fj13n98tsfvxshivryfyvxndxadx/kylmdyryssdzw/4tnohedzbts1lc8yjynsmfi1vs2zpreuzweuzi+h4ocl9i5ig19uuzayyqlnffdt3h0mi2vsk+1qsl03gsziar7rpe83irasglje0z/aob4yk6wqerp4nks4goyjekoqennfefckpaj151l114uazxuguyykohksa4rlkwch1dyweougd0uog1qlu1qpegxny4vomgumczzhjxmeeochbw+fequpkp/evjoyeqwsxqb1ymcfhpynibcvc4idr86znwlzbk4q1fubavhdd3vhfe/n7wxakzefhqyr3amhsbbmitiew6sse9bifkmihngunkjb7yzpg5vw6/okyx6bwrmyfrso2v3mkbo7qangjjgkcgcpb2fxglbbpfb4vodm8fz8z3hhbmmlplv7gjirbwcqnhxrvqocurviu8f07hgl5fam1cnn7tjg22rtnt9tttpdyiasfxywwbm2r8r7mxujzqhtfvzl4bojpwm/jttztjlrcmhkpyv2vgvi1oisondcihzstlgtgqttpds7e4us5ymfa0ckjogkhudzhewj5bihbtfqywjevypjka3p7hmg13glgg+jnuk7orgsxfbwlvmicnbmmzf/wqjyekd9aewzddfai0ndkqbambboxvodyzwisodpht1pw4bdsdhjqo7djrpujaupyqcwwrk4f9m8bmglbttig8gxburehh77yqnz7sovybbkqidjwkwyu9xx0gp3tdeiqywe7vzl9l+gatozcelds8halzfncuoux+wfd9pkh/oerawwfgxjposbsichtgpdxjsbbb9dobeuqxidku1oiezac41qvlg1x7iaixfuslef0dmwdtmxpov2i5uwdfhxa++hhkifowxtdkpyvy2gzxzulvap1za1g+ik38hemkfkpipspdjybauhet2uyxtkqlhio/yt/+x3b4vbhkgj0wuodvyqftsnsvj968kordfbbrvwm0k+egumwxzuowaivf4/yal2n03xhvplogq57qoowrjre28tdj8/e/ws/atrsyqalqyx6n+7qz3fnouagkaxxuorgxaysnaxwdit4pwiae7qmhtu3gfphdi39grmf4mufy4kiqzbqyte2udjxingtqv7znjyj0j5q5vkupdn2iegudkblrviyxymmumhgiwmozd9cjnw9rgxarumwvl1uttfxdxs6s50wmv1doheqw64ulednl13al9zo2adtb3wu9pa8xd/d3wubyrlzjmrjazwplu8tczbcxiamrm5ygrlllpzwkbmxg4y2qpsgtlt28j6wnping4yme0ujoefd/qhezzsiq7bhw97oxs+7jcd6jp2rkbqd5as3cg8isnmf0befenu0de/n3aqv3powe/xx5iirucshxoysis2h/gfpesr8iudfggocc8aoscvusv9bqvcqqxok1yifqnkbiiv3gcaorvzovhrkkdcgongazzvl5wmqx6wuq4eko1vebgn8zjhmk9fw3kcagdusyzsy6zyz3mtdm55ec5avxy8zcbz/xvxa1a8prl3aiqwivfztxfpcy9prjkll4yqs8kkxppred0fg4qkmcqbjquew+jmy7iglt2gxcaygkb2gkqew0knafbvrndrlvdgnlswuknyeykmtw7wtrq9odgeiku7u1scwuiaw1ci7kqydjhwmeaurkfulxgrljcxdjxfujpvffq12uro6snroppysrqoqlgsvejjbourlcqdspi3vdj1ojzkkqas3kmstppexlzm8ehn2ztjwkzujpqrvmfphlr0ba1y7aa4ywdqp8styea1kxphkc9ol3tpvozok8fapblyzzmhibtcyw9dwxpcmkbbya6pq0gzxbmoakickndufyeh/rgw1zwhqoygdaigza7md1godrgrkxeow2jtrlzrw4wt88dkkxpqyb0yjkrsuj1infq5gwzzxmlmqbkmlgiqf9uqaz7zgoku+drckzoyt5iqonxc1bkdwik5tplowosf8wkdfg07fswddg8ghajzq0inghl7rphwfjs+8pc40wsihheincpqlgstbzpg9ja59wpgzhhuzpxg+flpgv0wnteqk1fbkjleuldg5om/vmhtyu1y8invrzwt19lgpjo5wr7mksskczjurnpkxz8noekc66ms61lsmdzkky2ja57c/jqrzcet3ui2qin3hy09cxnrkiu5jn/zuub425t5bl1eszooejbttlixzec2wvqfvd5uap/rovdc3ms4upbrflpchu3y+ap762ged+obislx25p+twmyshedxz4mpg9q3vr3wkulmsnruobmxzichxerhxhhh0hz9jlvnnauzppqmu5amalanbrcxnjnpwuciizjn0eeyzy6ihuocnwils3ciaqgbbn1/vqog3wf2ch3modpjxyc2uos+u3bmrxiyrrbzhoreezsrxe4gjgulh0mihaxgcctbgswiczyi0wwtwj41vsl+8ju6cp6sijusxcfzqlkfbk1j5y7vijy79vjvtk+6rdbj/+xdrtn7uyc27dibt5eku1qvdwsbox6g8t25yrkjk290prxljvvsdyvmwfu4p75lqecrbdn9mxmqeiiew6r4lhxpooxqe0oxdosb6lfqo/xdu5g4jlb4km03xvc6u61w6hw61wwvmy8jrovsg/b034kcjllgcmijoqelntzmf895lyywtrg18w+dkv6zt1z/48n3ls0p4gkruibpj82v7hnc4cspvq08qjvmwmbd0a5xwczoq9rh6zfl/nf/3tthrz5btpmsv5qmreyj0lmngxdq5wsnjbme+wpoucwvj86hxgrrppnedqfgted3qicucvzyufijxufm7iaanxqqqio7bfgqz+w9cyskrsnyvurx9ii5xjtte9nl7qpnn5p2pdtfcd8mhuxxtxxmw/sny8lgxmdn1cnoltmpwuz7o3emjnrumbcqdmma3gg1366xta6a15ohogxmzctg/edz0zfa2u2w9umdt9irnkj+tive6oewpoqackci/qpzbfsq8fvknfhe/fdtpbgz8t7pc+gy5xhgku3hgjvdb9+rl7rup9+qiez6pkw5qolu6zrdekqgivnxv7tfhw6e/xtt7yn/wfpprew06tlwt6ab9x2atzsu84cqspt8eewcfox4amzyvlmdgy7jqvc3bevtulecnyjpztlondozzox5sje7tun6waw5xgaoilf1xfko56pk1zhyxzygmxjniodqqn3kmo1l2hlx0eckkkm1gjeia3v60vcb79g93lw/mnjatpxdmc4h7lruvz8vzbwfkzohkzvqel//capnemvhvwv5zvmppaqwuyrydwzfux9cz6tbppty6yiferdakffwht+u0dhymfpf5lv6xwpk1uaxhksxbzyvrrefp5cpmikbuhlcu0mz0kc0+g5lolqgudpcmmvh4zqlyg4qrwd6rfnutplrcseb+cao+c6o/t1h5jgmvcpkrvmhzafwzjomozatytsfbjt/nnrfkirbne0an7x1xhzcskoekdz2omvpc9yoza0+qsppk2ooru9ejjzhkydlzrgh6u6hjsheo3uwp6pcg+csti/7gce64oeqf1zo8bhbew6a94+qjq/nqmr5vdvmaygze/183c3ruafyngfx9/bg+3vjb7628znhzd/pvpo2h/w/rs2vtza+/vaz2tp147wxq6ft/53e/32+7d/d//39/9x/5/3n2ro7vsnzftrvz/7x/0fykp43q==</latexit> <latexit sha1_base64="pim2buniwj2/wyvvvvkikbweovg=">aaa0rxicnvtbc922ezz7tdvb0j72ha2kqrrlio7jpjlkpbpfvu1ohfw15dxesqosodyqedmanosz9lf0z/wv9km7am8hcycshm3yiobvf4vf7mjxuvaktmidg//e+8lpf/bzx/zynv+t//o3v/3d79997w9fi7zkix0v5knovw2ioanl6cvjzek/ltglazdqb4lrx9j+zzrywflsvc4kep6sognjfhijvzfv/mcrs4rrj596wc7eipyqbh3rzsw7qlzf79teiy/2ezqw24gu2htb5x3pf2v6wv09gtuxlz9azxg5rkcxrr1voj7vk+yby6vlt89zpje76yfb/psgsprxfsoitxcw9c66n+vzmqaur1++u3gwf6b+pptj1hxsrdu/j5fvfxjpj/kwtgkmw4qicty6kor5rbhkyulrdb8utcdhnynpgxxmjkxivflqq70tqim8cc7hxyy9vdulqegqxcinajksorfmg1a62s5kof7kvgjzuuqahbqjczl4mvdwlryicrh8soapenigsnowa5yeemzsvddnqrkysszvdawsdc7dpc36ttenxysf835twias8xzwrw3zkctimwf96nekfmtm9mspza/lrfkdmwyci7ewaqmz2i+m2mospuwnfx/xw3habxnuezmqdojnyjscbxocieecl6byrmmmcpvqtnnowz6d2pweoun7daimxiw2r4st3qviuutf9ofn6e5cwyonwi6soif5maqpjkehrqczihzgkjjokhewpzjala6iflycmkjumkml78poscknov9qwnc5zbgilxezmfmlrugzikervktymhlmir9nsvlbwhnkked9of4hfwmtshy+apgo+syros2jm5pnqzqmhapqpk+8lpmy1juf0jhllzpcdp8agg8ioabmewrmnkl8nxw0qksdmtbdbkuwyxh4thww/5hzctemjrw6qhtqn0srzidr2funbrizlqxev+ma70gj1pnxryzfmj76bhre+dp1ybqqickpjmrsaev4z97gynvb+/lk6xmjjrwxjii35hj8+plceozwcpifvtjuqzfwmpxno2ojszctrh3/7gzurbh15ohmvkl6fhvljzrl47o6ou6hvnpqiz/egefrpx4rvi9iwsbyyo0tqh/pgeom//j1cypenah0nfgo2qfs59hr/x/7ni6rjzhfqiwkwria7iwgv/jllc+wwglz73p+xkedcs/ncmhqk59ltkbyylbqq1pmfw4py8jcyb5dxc8nqu4qnvjlfgupi5baxpc15f1hmllgntmejttzk6vknuyrtfp0ewihayjrxf0jdoqbsckqeqgb50mei2geodcectfdbhffuopaf15u910i0fwhsmdciprbsqa4fo/gojobft87ymos4ydvwprwqgtd2fiiomyzw8xhegoep97x4ehj515bcso/tdumpgqrizfone9ygshnaqhs5rzi4pexzjq0lajxcoqqukz8hu/2wvvbm+8mqjgrbyzkiu3uqyqlzhfpgmfsyqlojishbmm4v07rohex9i1n+pvlxtdty6wjmb4nh+w+po7z/druy0ccegsw1lezearyrr54pf9a83h7vrvysdvygunwvibozmgzcmddtk8qh1qt0tqxpbveqjm8sfx4etso+lzae83pla32ejgj6wuhbhcu7rhx3ubgahpati++on5+sjttt2rnzkkmgggzrhtrnrdmzfpqqudosnfcigl3qpc0nxrnz2krtxywyacilcsctbaom5apgvwipc0gkkym6hckoax1xqmd9dpuefbttczpryewtqseznieqq6oasbmt1guswkoh6aosyeaadfpoavakwywrxiqc7kgeen8ybd7aqtwcdzmxmhx2fgxqrh0qqwuvlgw93kahwb5yekugaxodvmnjvtbj0vibld5syylewizxyqvnekog1z7o/0rts0kblcqo/8ffzjohbmaxkmger1ptlci+gl/y9ziot/mcytkys0ykkwja2gpd20hhr1oagk+hkjjxvlxbiobnecmtbya1xy1ar5ynupi3gsvcqozlrtmlnwk7uyqlrrw0wbedz2qpnpk2hlu5xobdzvtqlzsm69hagbrsb+diwwclegv2xqwsqtwq/w6kzcpliwwxebtw/7z794kgzhqe6lxhbu4vct2gpkk+tafuyuug91qrtatfcqkgiyt3eynekv8f5af7g+arju0lnquctsvulikoibewui+eh70cv2rubmqmi6mmep/u0gdxttrmojaf8bqerswsrdwl1gye+kcigbo0jh07txn6yqyn/yg5ledffcucikswamexnrnscyp4e0l+85ywcdi+auvsrqql9onhrgymy0b4sscjjldbxosjjmxfqozvvax1wevjsffdb00xt3v7orodmdfbq6iremoulizazzdd2i/cztga7qd8lvawvfw/3dzlawezwstg4wm8kzvplpmwqlygpktowbksyyz2vpg5syognkkbbrzm9vi+ljtyj4ngjntlo6bnw76orm87iquwz8oe6mbpuyqg+o6daygag+qetwopcup5n9g3hrdbta3vzd2kfd6tlhp11+aoq1aricageilnofxpt3rk/cva3xljgnpgjrhfbkr/w6lxekstpncibajsgyil9xngjq1czrx65ja3bjjrgm2vdecjqseslquhintb3m4dfgyxzjpx055agia7rgm7gomcs95k3zuexguql18vgqgc/11dqdwvd57cg4kmirxrbxxqhgpauy5c5emklpjjmat0xndh4oejjnkgy6lhspibmo4hzu9hsxagbpg9hikhsncjqh21utq25vxyj5uslpdchciprco8luavaaxnoirg0vugsfcaftqio5egayr1jbglkyn1jv4ky53mq48x1cavx30ndrq6obj0tqaweq6japrkrxbsqalm5qkhuqsd1xsbai2spkmktypkraaxszwtpbitds7svis7oz0evth6ys69gwnwowcomlnud/llwbanssaysnpajd0376m6cvh2qqs2gwtisg7qmfvxvl6qppaw8goqunim12zdmpigpdq1h2bb/6xsnc8b6qgbgwcim2ezpdqqha4kysrdsno7ubc6lufrfpayisaugadgcsq0ridsdx6hyfmwctc5qgsjc4iqn/ukms+2xjpfpncxcmtsreyqqj13nw5hcccuuzyzlqlbfmcgxrtoxbmhxypibwi2unijyb5e0lx1hy0vvdwunfrcirxcdqj6prre22aypsqofckygyyvgacrvhzarldfjbrkitxwsoyxfc3yoajv75ou8lnwpip1uwce9fzyky6ocmezckrjamsbq56yswfpznphoupkuts0pnc5cmniqoe3p2uk83bld7inqiddx2npxctuziloyzf2vfaenubeqzdrlgtqbcw7bzyl2qnnsfubbw+bmqf6zlqwt5rofd20xyz3j1m2owqe+tohg/jgsepv9uafk8jmeoxg8c+jqypun1a91pfczlj0vkgzmc4gh8xqx14yr9ic/szvttqlmt6udluqdac2p20xmtyzavlaoigsz9hnss2oiibqhdccjutwigqog2zdf76pbt8n9jb9zjnascwhnvqevrtwtk8yskuqcx6exhgbk13uicyli4djohwmyharw4elias2itflrci+nb7jfvcbugqerio1esqn2ai5bqznsece1yooojb44hzpuq3wyf/sq60ze1mnnu3ygu+xpfnar3cem6mehve9uwesiznvdkuczy1b0g2r5sbbuke+t6ngkww5/tmtebcihmokejr16tqmahndl3alg+i30dvt3buyocswecjnn8bwurutcoovutcml5lpcazluhvwdn+jhi9zrgjiiakhczu8zcsvqywrja28twzj0orxlnp3p9za/euzwkgazfqgompzrvu0xygzckvjtzq+25c4b1qzvfbhs5r1chrd8t80f/bladvftsmc6yxwkyezkoqoq1b11alk40l80r5qs45jnwpzmheqqvmgz7oz8znz3ccj27jxfgqumndwaysgbqfwqqaa/sf+ncpzb0xsyrfo/i+8fha5fxiiump6vl3kcb7cfu+rq+y33ijtrjlouanvfoxjdcv8+bqrgfl7gdh1r3rgyzzxrdmkhwaysdo8nppt9hgd8sljup9ihoydso+48yoagdndriawe4bysgviv/mzulva0sfii1zyfe/leycffukhcx6ep+jmk09+nnyjuyfdiwzdgp1lhctmu7dz9vrxfftt/ukuz0tplxhydc0qkecercwqdiv3a9c925++om38t/iyefwcnq1tnhb863axojbespzqhw4ht7mik7/kjw1u1nenm5sp2sbu/v2ttajscf3hl/r6mchn0/gmin7u1c3pipblmcbgiz+w18oj3qmrnadj3fgaxapcw52bixcs6rvviqthwdxqkyyuit5hbs+rxdxvf2c3cvhi6dh1aavgybxd3pxoqz4qizhlzjqouyrgjl++ffsatibo7hdyhuztcehgo1wkqtnlcr7ezxb2ftblk1f+isg0ek/gin8z4golcoelisr1zlsvqo0iouku26xqjsi/sk5fsji3mkxfdpnxpa4ptelxbkhbmgf4yzfphqhx0jwuysg1w+bc3xkwfc8dsa1d8f1dw/qpisnyq4eiivmawa+tjdrdqybagnz/3c3+aejliqick5p1fyuq9pm40xq9y7mvacref7kqi1p9vltev6ddgp78ddnovg7x3taj1d1pwhc8dqpht1x5r4ek3b+3cg80bu9ulvmdobp2sp117y9j+e8hjyirprfn8ewgda/8y9mruafyngfxz/yhx3sj/71copzt5o/n3ln7u9rf1nbxhut/w3t87vnaydrr9bctf/d8+7t3hv/4qcpxz30h15o6e/untr/xov9piz/dws7ovw=</latexit> <latexit sha1_base64="wu5pkq/f1oibhpdr2sfc7aj8t44=">aaa0t3icnvt7c9y2ezf7tnvx0v7zf9hkmtqxpoocp03b8uwu27u7cvtvj7xewqosob4kkqab8hrnhv0o/v79nt0feecsacu5mrffal9dlba7i8vduzkxqq4o/nfrbz/80y9/8tp3frb581/88le/fv+d33wpesvi+irmgrdfr0tsjbx0lwiqo1+xgpi8yuhx0cvdbp9qtovkvhipliu9zulascmliykqs/f/u1nk5estvwufz3izz0ixufp2jl2ub7+50wqpgjtm6ftdjkzv3gqq7wahrpkz+vzbphldhxl0lpczghqbjvgk6lmzxr49o199h4klm3vnc+f4djinsaoxtzizjdcfzxnnc1w9gra8qpkiis2z97co9g/0t+b+tnqpry325/jsg49uhqmpq5wwks6ilcetg1kd1kqofmcuefeslis+ick9gc+c5fse1lqrtbadnukw5ql+fsrqtx2kmurslvmikdlrm2m3yawv7ars009oa1aulajfbdqavlmgeiczeirmwnizjxyqwdcqnyc5ecrwmhebg25kkshcfxwwlfrrhyp5xg5ru0hkgysxw9q8yhqt/hjyg/m5k1lfi2h1nae+vaggtrtmhwqiqmuzbi7kvfq1ysvamow6yirm7c39cddyc8bvfe8hlydoljiroqvbdcqqgzhbrnulpyumhko754kczuenpqlqlg/rzbvnwwqpfid8ucw4qsuogsfownjyi03ajrkuw/zm8nvw0cor00sil2aqooiycta9mnoijyisrhwqym4yq5uywk5ipwzclckruwlmcmqriiqyrzkypexc0aheuhwypstlpwpvh2ctgtkuoh5qr1tljnyfhy9apqj/hjtoipssfnmqpuchqtvpk28qjmdnhuy0zuwtjxcdqqrhcbfqakyzbxoa1ageolo29qhnm36djrhkcfhmfbd/sd0kcscnajpobtcqjini4yjsqoecjnli1mzqcyb35atqwosyxw8yt3myoa1d4zowdxwuvrrhnq2mcoktygss7o19fvzktnxgmktb5dtyotp8ceoiqwqfyq+fb5vwsatm+vbji6uztzneffv0alrcefxo8dwosvf46poeazgntf34aid64okjfpzibg3m+gvcpyd5cqulm04ifyidx+3wxzskbr1ta+gb8fg0q63ox2/2/xmkoq23jg6rqiwje3q6ixt8klspeo3vwshmnwgtme9vbkhqqftb/llwmsigi1vx1tebcstzexopbgw1v5bitavz4jdffkrmwo4s3zwud8fjsjw6zhugbm9plpnfzhoadelmbeicede75hmyncnjhvulqqpnwyklic/qg6km2c6cudkrzgm+x9d3fqjqxa+yz0mswscyprl86+ggm1sw33ri40rgipxclne6bmpzcri6zqxdheggu8hz4ojw8ogzocqlfx9z1qymbculnwit+tibgys8brzqaxhw2zpmtfqrgctetv2e1whl9/yy+c5y3bmbiimhdgkvu9omksyzq2qixkmvikcriouqdonco0wl3l3rtzyrnmc1wwswcczk/i4ciruyrbz3c1unpxfifmfs383is8dudmdtxrkap7cxu8s7bhtbmtm6fmsllaxt4aypdnxlmkplnnumzzdyaoe3qr4+68yb307ropmzttubalnks+cads6mpsi+2n6abi+y9umvjp4dr0z7hdnzc1livkcmaln0562qlx0oowsvpowg5ux4iujatbq3m7ksoj5ysabekrlpogiubcicqc4r0w4dftzgdjtsgtlm9t0l9cbtepbttwz5tyeotusrx3krqa6oasbmt1iukwkoh6goijeaalfpbavajwywnxhqcrkwkzj8ybl7eqfwcdzlqqpp3foxrnh0qswuvjgw92loy4c8yglolayvtvmvjgzbjyribtd5syelgwizxzqvneoog9z7k/0jzr0kbldqq/8llzjoxdmax0agerftt1cm+wl/u9ziot/yjcwvdbymreaegbbhh64dw140gwqfq6g1lul4g5fnzsbgogunbj5fn5/7mifxbrnxjthzlguzfz6n9io1fi34yxy+dd2qpgvkuhnu5ey2gjbb1bny34owqr1ez8mozo4juqj0zbvapj5ya9wbtly2ua7ceru17/jnshsndojabyhzc1zgsq3yauyy+msfri+6hnbrtdpo8peoyhi2cbs1qqzz/1ewsl9pu+7rotjryg3xqmwyhnp4z6h77nnjv+hfrzp10xyxg9h/uw36lkzd1zqeurbwj9sykkwzvqsk2rdvfafwhszkcuchsy+u+bfxmd8fptj3urdpde4njm4+tzfbaw9b2deow3pb2l+dvnkhpdm+mcavuwzzgf/mycw8ojbgoencdsnswdnexkru2gsv64uvke7pzl93tgmur3najbpzwownhbdtukf7jd8bw6drgn80zoqh+7+roywghcobxwfkglet4qljdl6q18i8ybejw2gzy9yxmj92xmjwynvinrpgnstarvz0xmhcd/em/owoh3rb467og/zlcw/0w8cdclrdlz0rqn4d5aq3ck9ijvmfltelkbsmze+tg+s1njm20pxntmhrebugrgbsoe1hfo7o+hp87ggf22pcs4becqx3pd+dlcui02ngpewwumqheok/0zcpndej31qe4syunxqwqboqnfv3gnvn/ubrfyfabujaalye5poqkbaaci9lzr+xovdctgnntodnajtg0xudxc3x+q1yiebk3qmvwgjgr+ute5p7sgsqwlxihjxtjimejc6a4rgujmlujxsszbiwnxvdnazcmsxhxtaycsegzbiwegywryasxq+8evokkju9kpsqmbn1gk/16cvn8urc3y2ieiwweticqkwn0mkitjyxomqdpa7fw3o5ixhg+ylwro5jqnfor1dpjthrzfhsdvstktqvsrjyxkuk5vgsuqmsrho1vzkylerplyzvddk2cozhasbesczsfwdkjqb6tif1pwdnwhixwbm/hppl7gajaljrbeu8oh3sffcyyqgcz5nkyjdnxotsc4w99wuzcgkdxsf7pb4h7xtnoaghgujiw/clhvin0l3zpkgggjaigdr7iu+hun1w11yipybr2y281rdkv+erk1e0omg5mmh0pxu7krv0cwvniotf7velgf0u1f9sdnz8sjuxkfkpvvjm1wax1uxnukt3o4afe5kuvbuwc/yfbim7dizyo+wbqhoqaobqbghz9ojirs1ofbngcwpqekjreois6foubbrpgtfdcrguwmwyqmi6jqilyjg6im6jvvgtg1viiafvwewtf3prppobubjzaoo4j76zwuj3coi92fnwkhjl/vz12ik/tzjphevpku9s0pvc8jymxappbn7klxnwi3e4derhfudjz/xelbqzh/f7o4qi6hqzz6dljpusldkx3q7rk+zyjzjrgdfezd1zpq9kunsnd8o9mm+xu+mzo1t37xzn42e8mctvelsz7ckdu+lj8njhxgwbulvnxqvsqdy5wsqy+vhh8nnyvqmv3qnpgt6zjgbghwwfyqy9cezat5ooi5z6uxrmaneygflgxhxredlahedgqooqhlel7zqd89dh/ed7ya2+r1zkvvhw5kplmrchq/ckjvseetmiwjfe1xqpc4jp49jjgggfewjc+kzgbckcu1guvepat9kvurd2y1frs2wpx4ketjcvomiopehcc0lbwdaed+z2ybddoflptabtb2ds7t6klfh4rbwp9wpluduzbbc7lxm6mbt1snmmwng+idm+aae4jxvmu4jklscem7azaqwjtdxgikdds3raole59gkl3ka/kd6v7dzdwcedxudrfjce9d1rj1gj17loes14bgzrkd6bp/ti5glomeyktjyzwernyvh9vq9yrwzjs+yehyk9tp5f+pswhlzy00dymqmamx/ba9y2pudfcipbeft3zytaoyau8eggywv0iet3q3wx/m6wb29+uwxzapzatitso5afjtvx0cunjswlthuqytms1y8uyms6q/yznslnpm1nnwjnfsl8wbdsyh3bpiqbgp9a6oad+wx4mi9lgynlfewjcv4uslzkqmgkgejp0ver1vsj975tcfhccyo2voy65q2783eksk/zfvpez8fuz2nxvtubjntdl0xqddphic3w2s+0uez3zqunq/mie7j1k77jzoho4mx2vj7b/hvjhfeql4sqo/qbzqirliyxaj5k9sghfhyoev15/+m4bwl42qkel0roggcyxhqsiey1/yef69e6/qef+kmmfizmhcbcrmnsgdgkynfbsd/4x4xuxf30e2/jv8etz53v9bqzslf2w7e9alcmtloa6na7ffgkgv5rbnqmct6eztzoyrj3991rbzkqeh+xj025crxm82stowqwe/vdehhmetwgablx9yxy6gfqgtfzga82yumuc7azkofeuq/mfwz5bigdmlwrxsxrpa1ds4ib7rfsxg6fp3lcb4egarj3ohcjypqvzhlegqldwlrkkjmp7ywlib2yg+ulvjljhbii2ghlxd9r0d6f4dnkdti1bgvcrzsgfvoftog/snqsdnu6mkxf5yhv69iovpdiospq0ij2s+ziesnpb19vgdd5ttkujs9xsyqsbh+gg371yij+ccfnrxjuvg3m+psxquuagwvuguvuhjrdsj6kqj9e1pgdwpzzwsqxkqsqfwt/o932n4myeioguoz1+7upx7yfj5lqdzsna1gpemmlxtkak7bytd7u1q7gmrccrf0se+ch67obnkn47dtbnunyaijjuljoyz6biggow3n7wedd5azv3j7ecj5chthu7e+rybgm29/4rzmt+09k3i8v7+1pdvyn/76/9ekn7z/pvlfxu40/bnzemgz8zeptjacbxxuvnujbp7r14a2pbt2//9f7z/en92cg+onblc1vnwy/99/8h25tpg4=</latexit> Noisy Channel Model for TranslaDng French (x) to English (y) from Noah Smith

12 Modeling for the Noisy Channel we need to model two probability distribudons: should favor fluent transladons should favor accurate/faithful transladons <latexit sha1_base64="pw8oji3ujegpi9vjlgbxuppo7ly=">aaa0ohicnvvrct22ezbtw6rekuzn2xk2kqzylkk6jpnmkvfmffu1o3euxzzzsshrqbkhbxjvbsbzmcp+7tp2ifoo3qv4dkkaloroxhybfltylhyxi4ucimfc7u//99zbp/v5l375q7d/vf6b3/7u /fivykof0ezgnof8+iiimlkppjzmj/b7glkrbqr8llh9g+3dtygxlsxo5kohzsukmjvlijfsdv/ofrswprp986mu5e4twqrl1rdfn7gw1/ep27d33yj+hy7kd6krdevte8xxrpufvxf1r/blye+gsiucvnna19x3us3qffx1+sfz2oysn8vb68by/pagsfvxtdt/lszinkf8/f2djf29f/xj2x6j52fhrfo7p3/3wlh/lyznstiyjeej0tf/is4pwycke1ut+kwhbwkss01p4zehkxvmlnfz7w1ateeocw79meqq2s1grvihfggayjxiizdasdlwdlnl8yvnfsqkunat1r+my8wtuofq9iheyflkadxjybrj6ohroqgmttn7rpiajtfkw94zsstqxyz4w/dqyk2lcwnm/ni0tyxg+69eg+zrlscyzfvu4bb5ljvu1foahcikpwrkgzsvyglxrde0wru0xsvp2mn6w46044lcm97wzcdrxjmrkweg8acqgjhdqoam0u4r9dac5p2dcqw1ogpyk/tprbmmwm6pfce9v8vyiyvvau/qgoa2rrmbhszzd/ezsu8bqa6ptgrelmbucdmg4mb5mbryhrapbdhkkn5mhkvdhp6suk5wvkoe7hoyixmuidjipkqkgwctqcj5reza8imu/m1n5yk0jmhsgflcvkiuevlkox7s4x33m1dcw0rnnpibkhg5vndmnxpvmtorkd2jmskpnlnp8apegevaapgkfc5puvpo6wttvpk3rbomimleifkba3/vibiuakxiohdq9qe+isobdkoyqhgkmwutwr6txghekbor8yylgbxhpfto6q3ztojanvzcufec19rryvfnvy+tt7n55/ois0caks0beg3i5onh2zgldoaxjd55u6jhiomz65ugr0dnrcajepl4afvfepty8glwqhp9f0ypn0riudo96qk8eweinmubqwo9fhnvtkhawgljjhndhc+c46t97vckgn00afqu+inao1hn4au/fpqfjamnkssqlgjcupjulwa98gcuxwk2erhc8p+lttoxncwu0tzxpmicjbdbytvutsz63lgvhlmspoelneyf3fbv8couclbflbei7mvlombkpy5tsv5ptuslsmk/zicz9ugmbkade9y7+bab1wfhhvynqwpmkwsuwt9abgosylok4iilfrt9fvndccnbcb7jrqikawbkmulxwcfdndysxdvkw5dhittcllsrbcla8ii5zxjdzed6y56l3dhdw4ilxkilyt201gynbymimwqc4vmdierocwzmhopjinfmtlkxgckfdfeev3/ddxng/evpba1bkziehf2k36ygsbboinmewqeqenjlfeijhncunana7s/rgjvz6vgky6wxwhezfhen2b7pf7e5qqrejdgkcworbwtwb7fyppj4vohm8pd9z3lbb2ilp1r4gtmtbmqpnxbsvkoduldlumt073ki5ven14ek7dvi2wnvnt5st9hcyi+sfb2xwru0r8d7mxmhzwlqpvzp6crw07tdkz85jjhobmax00z63tmxauelntkjrqoojd0kqtj10z2dikbv8sgaaijqknlqqkjuqkyfciqqtbipmjoh3jdml9cu7buhv3clg22hn0o5olgsxpjzlpiichnnmzj+wkjyeud9auwzdnnbi0nakgfymlfrxvodydsls+znh99mw4bbszliq47cjllvj6ymykkywlo7zna8b8ozfkteynoimtkzotx+vkn2u8pimsyyemo5vkjvfhqetdkd7i5pasnhuvub/cyownqknnlyedpry05ygrphf/se4kun/zbmwlcjamcszegbchh3ydgx70qqsfayfxrqk4g1gnjmbggktnkr5ytv84wqgxlnjxhvgz1s2zfz4fnqnvfo04ccnvb96ih3mlluzqmr1nho221g52gdew9qmopn+bxplbcmrkrl1ygmxribqdvmuuyglyymn2rf8s9+9fqzxoczerzk2cklw7dqksfw9c6mwxqe71vptjvlifdamw7ingibw+f8gc9jfnf13ac3pkos2k6bksurnvlxqffhk8dn6v6nmvtrdtgpu/3adootp1jufgs6m1sm2lgrhrs+xzcbeouuankax6dy5z9ijzw7sfcwvbikuxbrewintcym1z5omu8cbfvad5bbokpjxk5v0ic+1twxxztzbnssxorgzhw40wexylvsuzqzsyy8dkkycl9xl0hr3vboro9mbf9c5ibinoedixg7ydn2h/chtga3qdsafamvfw/3f1vewejarta4wmscbvvhymgcnyglktusaks2xzgzrgzkbo2bkk7bpzi9ti+tjtsn7pgbktrs4bn4y6ido8larugz/muynbviwqw6q68sxgqo9qepwo/cipjj/gxltdllb3/xe20fe6tlhpv1/ayq2areeamagldocxpf2rk/afw7whtkmpgvohffkrvs7lxijstpncifajcozilxwn2no1m7pxk9kanfjjbsn2frdcpqsesdqurqvtl/l4tbeyx7lph0f5qkialjhmrkpmco9503auexhuqd18vgsgcz118unwph69o6zkcaoxi+rjbuke0wzylm4zisctsyxnozo6/4yjctjva50lpuyfjcbwzws7desbsbqmllhipuyfmomsk8mgl6vvalzpilvhzkbedg9dycv1egfjffexf0iohimfglygkj0j8jgilsgmajkpawuxftxuylx4pmc0qzqo6/rvkdxt47w0crs0nvujzlkfuoywfyljolqkrypkq4ttvgsnjxktpw01fqytmkcr2ra3kncynvnpi+gok4n1kvna8bib8bjpop6ww4ba6pz0qhleua7phv2zurfodymlcaumwwj2ruke+vk0hpsbnpcdkgdqprpmnvqqxkehkbuczzwj4w95jlqntrgeabpdmw+i0k1w11zixayrms34lrfkmydb1ze0yac68aguzxg7usq0u8nnmsifjzhlwb0qvd9q1zz8+ngskfiz+zkmzs9xvqvreouzu0afsxlknfwwiyyfxikanuxyo6wewdhqmgeqmoezntdire2opgfh8enxkmgjeoggvgnxjto0wsjh8y5vwazw6ay4ztcz0rl6lkwivzzuqpwksgguguzt/z1rztkbskst6ku4py4zgyz1ckp9mbowsfjkhrz1wmr/jzksedyt5vcl5lo5czpauwc8otmp1xpdm7xbrdbhbzraoyjm4islmlh/m6oaslb3swz6ckknusfsgy3xboko7yjpirg9xft+1zpgrl2xsobso/mm+xuemlov91br/n4ga8msbna1vr7ssbuuhg89jexfydq1rrxklqqsy6wmmq+xhh8ufjtwav7qjqjz6yigxyh04escqeceddupouixm4ulrnanez6ppjctter2ucf40aobbeouqvtm63z14fd4pvqdr5hou87sedivpfqbw+w4rvlpghi1ovam2jrvcmfxhrx6tbbhisjbg51i7aeycfellk8rcc33i/zf3zdv9gtrafegvg0eywnykg54dy1qsf+3x73zfzrv906+y91og1uz+lcvhur8pgkbflr5zzwy9tbilyvj1qyaeqvpoyzrhldsnnzdhat98x3obusgj2nz2ycehkzciguj7omvy9bbxlo0mz5e/0ger+2cwcdlwwgh6x53hbwd68drrva54lxnemxma1eegoe9ioryyndgbexusrkoz6hyfv5tws1ti1vmx0civtkeub+xxu4d2loa0mlhajmf2yucjvmacvlqwjkud83ewlvghkodzj0xqmowx9c5ov+j6y8eppbzphjvdriimqehcxp2lqgkllpljknyln1zmgsfnqoi1zdnc/wdd4zbjq7ethxs+bcgpaa6wo2zqhu+nrsbrzyl8chfizrjvkikrqv93eppuxojaaynpoudr9pvj+w79v6gpk1n2kla65jxts7h0sc8zpvrk50tsx+fmbda71hmtd1wyrspxkg4gyv/xslbxtxvna40c8yivs34jvojiog1myhqxnsvpgm8hlyzv4mvd2w9chskguw9vpjlthxr8broo/3pgrt2oofle9wxkdhommwvm8j0arppvxcvdz1p/1utzlvm2hkfrz2tamaxcebc5ukvdr9knt36qefebv/e558bi2nv8uevtxfqu0zukwnnavuh9vhgyzi9g/cuzov5c3pzhzkfri7b19rk4gu8vbwkxodoptmyvhzzg/36oz0mjjyoedqxk/rc+vrz9qvrumxdmza4nhowc6ghhtjtzqxsoxjia7ivazqma+q1vf1iq63x7b7oxj66lda9dudmo5h7lqufv+vjbwbezqharvq0h8/supnemuh10t5jzmpjesw0upy9axfex+czyubftuyqqgf0hba6fcwhv8x6fg57olcsnqdi2wlsonytrllfqtkg9gvksuhjmqtffmh3eqpiwmapculeuoq/blu+ox9efolbde1yld9tjlxp4x1xemaxhmhxhdnv+5d0ijm6igywuwd5mmle13gekiwkl0pd5p/ospckilgmlbn77o6at5obvxvam56silnrs7uklafnpvn1xgntgcpc56dtfnfb/xgvdedjhsvnvrye1xuqtaj55cdzfnd0qo1a24h+lctxhfn2wcyzmpii+qq+x0vdinh8xv/agzk/omm/fht3b3r/t7om3sbn3/s/pnm22t/wnt/bxtttpaptc/xhq8drz1fc9f+d+u9w3++9zd77997fo/re99o6fu3gpr31no/9178h/jgm6y=</latexit> <latexit sha1_base64="pim2buniwj2/wyvvvvkikbweovg=">aaa0rxicnvtbc922ezz7tdvb0j72ha2kqrrlio7jpjlkpbpfvu1ohfw15dxesqosodyqedmanosz9lf0z/wv9km7am8hcycshm3yiobvf4vf7mjxuvaktmidg//e+8lpf/bzx/zynv+t//o3v/3d79997w9fi7zkix0v5knovw2ioanl6cvjzek/ltglazdqb4lrx9j+zzrywflsvc4kep6sognjfhijvzfv/mcrs4rrj596wc7eipyqbh3rzsw7qlzf79teiy/2ezqw24gu2htb5x3pf2v6wv09gtuxlz9azxg5rkcxrr1voj7vk+yby6vlt89zpje76yfb/psgsprxfsoitxcw9c66n+vzmqaur1++u3gwf6b+pptj1hxsrdu/j5fvfxjpj/kwtgkmw4qicty6kor5rbhkyulrdb8utcdhnynpgxxmjkxivflqq70tqim8cc7hxyy9vdulqegqxcinajksorfmg1a62s5kof7kvgjzuuqahbqjczl4mvdwlryicrh8soapenigsnowa5yeemzsvddnqrkysszvdawsdc7dpc36ttenxysf835twias8xzwrw3zkctimwf96nekfmtm9mspza/lrfkdmwyci7ewaqmz2i+m2mospuwnfx/xw3habxnuezmqdojnyjscbxocieecl6byrmmmcpvqtnnowz6d2pweoun7daimxiw2r4st3qviuutf9ofn6e5cwyonwi6soif5maqpjkehrqczihzgkjjokhewpzjala6iflycmkjumkml78poscknov9qwnc5zbgilxezmfmlrugzikervktymhlmir9nsvlbwhnkked9of4hfwmtshy+apgo+syros2jm5pnqzqmhapqpk+8lpmy1juf0jhllzpcdp8agg8ioabmewrmnkl8nxw0qksdmtbdbkuwyxh4thww/5hzctemjrw6qhtqn0srzidr2funbrizlqxev+ma70gj1pnxryzfmj76bhre+dp1ybqqickpjmrsaev4z97gynvb+/lk6xmjjrwxjii35hj8+plceozwcpifvtjuqzfwmpxno2ojszctrh3/7gzurbh15ohmvkl6fhvljzrl47o6ou6hvnpqiz/egefrpx4rvi9iwsbyyo0tqh/pgeom//j1cypenah0nfgo2qfs59hr/x/7ni6rjzhfqiwkwria7iwgv/jllc+wwglz73p+xkedcs/ncmhqk59ltkbyylbqq1pmfw4py8jcyb5dxc8nqu4qnvjlfgupi5baxpc15f1hmllgntmejttzk6vknuyrtfp0ewihayjrxf0jdoqbsckqeqgb50mei2geodcectfdbhffuopaf15u910i0fwhsmdciprbsqa4fo/gojobft87ymos4ydvwprwqgtd2fiiomyzw8xhegoep97x4ehj515bcso/tdumpgqrizfone9ygshnaqhs5rzi4pexzjq0lajxcoqqukz8hu/2wvvbm+8mqjgrbyzkiu3uqyqlzhfpgmfsyqlojishbmm4v07rohex9i1n+pvlxtdty6wjmb4nh+w+po7z/druy0ccegsw1lezearyrr54pf9a83h7vrvysdvygunwvibozmgzcmddtk8qh1qt0tqxpbveqjm8sfx4etso+lzae83pla32ejgj6wuhbhcu7rhx3ubgahpati++on5+sjttt2rnzkkmgggzrhtrnrdmzfpqqudosnfcigl3qpc0nxrnz2krtxywyacilcsctbaom5apgvwipc0gkkym6hckoax1xqmd9dpuefbttczpryewtqseznieqq6oasbmt1guswkoh6aosyeaadfpoavakwywrxiqc7kgeen8ybd7aqtwcdzmxmhx2fgxqrh0qqwuvlgw93kahwb5yekugaxodvmnjvtbj0vibld5syylewizxyqvnekog1z7o/0rts0kblcqo/8ffzjohbmaxkmger1ptlci+gl/y9ziot/mcytkys0ykkwja2gpd20hhr1oagk+hkjjxvlxbiobnecmtbya1xy1ar5ynupi3gsvcqozlrtmlnwk7uyqlrrw0wbedz2qpnpk2hlu5xobdzvtqlzsm69hagbrsb+diwwclegv2xqwsqtwq/w6kzcpliwwxebtw/7z794kgzhqe6lxhbu4vct2gpkk+tafuyuug91qrtatfcqkgiyt3eynekv8f5af7g+arju0lnquctsvulikoibewui+eh70cv2rubmqmi6mmep/u0gdxttrmojaf8bqerswsrdwl1gye+kcigbo0jh07txn6yqyn/yg5ledffcucikswamexnrnscyp4e0l+85ywcdi+auvsrqql9onhrgymy0b4sscjjldbxosjjmxfqozvvax1wevjsffdb00xt3v7orodmdfbq6iremoulizazzdd2i/cztga7qd8lvawvfw/3dzlawezwstg4wm8kzvplpmwqlygpktowbksyyz2vpg5syognkkbbrzm9vi+ljtyj4ngjntlo6bnw76orm87iquwz8oe6mbpuyqg+o6daygag+qetwopcup5n9g3hrdbta3vzd2kfd6tlhp11+aoq1aricageilnofxpt3rk/cva3xljgnpgjrhfbkr/w6lxekstpncibajsgyil9xngjq1czrx65ja3bjjrgm2vdecjqseslquhintb3m4dfgyxzjpx055agia7rgm7gomcs95k3zuexguql18vgqgc/11dqdwvd57cg4kmirxrbxxqhgpauy5c5emklpjjmat0xndh4oejjnkgy6lhspibmo4hzu9hsxagbpg9hikhsncjqh21utq25vxyj5uslpdchciprco8luavaaxnoirg0vugsfcaftqio5egayr1jbglkyn1jv4ky53mq48x1cavx30ndrq6obj0tqaweq6japrkrxbsqalm5qkhuqsd1xsbai2spkmktypkraaxszwtpbitds7svis7oz0evth6ys69gwnwowcomlnud/llwbanssaysnpajd0376m6cvh2qqs2gwtisg7qmfvxvl6qppaw8goqunim12zdmpigpdq1h2bb/6xsnc8b6qgbgwcim2ezpdqqha4kysrdsno7ubc6lufrfpayisaugadgcsq0ridsdx6hyfmwctc5qgsjc4iqn/ukms+2xjpfpncxcmtsreyqqj13nw5hcccuuzyzlqlbfmcgxrtoxbmhxypibwi2unijyb5e0lx1hy0vvdwunfrcirxcdqj6prre22aypsqofckygyyvgacrvhzarldfjbrkitxwsoyxfc3yoajv75ou8lnwpip1uwce9fzyky6ocmezckrjamsbq56yswfpznphoupkuts0pnc5cmniqoe3p2uk83bld7inqiddx2npxctuziloyzf2vfaenubeqzdrlgtqbcw7bzyl2qnnsfubbw+bmqf6zlqwt5rofd20xyz3j1m2owqe+tohg/jgsepv9uafk8jmeoxg8c+jqypun1a91pfczlj0vkgzmc4gh8xqx14yr9ic/szvttqlmt6udluqdac2p20xmtyzavlaoigsz9hnss2oiibqhdccjutwigqog2zdf76pbt8n9jb9zjnascwhnvqevrtwtk8yskuqcx6exhgbk13uicyli4djohwmyharw4elias2itflrci+nb7jfvcbugqerio1esqn2ai5bqznsece1yooojb44hzpuq3wyf/sq60ze1mnnu3ygu+xpfnar3cem6mehve9uwesiznvdkuczy1b0g2r5sbbuke+t6ngkww5/tmtebcihmokejr16tqmahndl3alg+i30dvt3buyocswecjnn8bwurutcoovutcml5lpcazluhvwdn+jhi9zrgjiiakhczu8zcsvqywrja28twzj0orxlnp3p9za/euzwkgazfqgompzrvu0xygzckvjtzq+25c4b1qzvfbhs5r1chrd8t80f/bladvftsmc6yxwkyezkoqoq1b11alk40l80r5qs45jnwpzmheqqvmgz7oz8znz3ccj27jxfgqumndwaysgbqfwqqaa/sf+ncpzb0xsyrfo/i+8fha5fxiiump6vl3kcb7cfu+rq+y33ijtrjlouanvfoxjdcv8+bqrgfl7gdh1r3rgyzzxrdmkhwaysdo8nppt9hgd8sljup9ihoydso+48yoagdndriawe4bysgviv/mzulva0sfii1zyfe/leycffukhcx6ep+jmk09+nnyjuyfdiwzdgp1lhctmu7dz9vrxfftt/ukuz0tplxhydc0qkecercwqdiv3a9c925++om38t/iyefwcnq1tnhb863axojbespzqhw4ht7mik7/kjw1u1nenm5sp2sbu/v2ttajscf3hl/r6mchn0/gmin7u1c3pipblmcbgiz+w18oj3qmrnadj3fgaxapcw52bixcs6rvviqthwdxqkyyuit5hbs+rxdxvf2c3cvhi6dh1aavgybxd3pxoqz4qizhlzjqouyrgjl++ffsatibo7hdyhuztcehgo1wkqtnlcr7ezxb2ftblk1f+isg0ek/gin8z4golcoelisr1zlsvqo0iouku26xqjsi/sk5fsji3mkxfdpnxpa4ptelxbkhbmgf4yzfphqhx0jwuysg1w+bc3xkwfc8dsa1d8f1dw/qpisnyq4eiivmawa+tjdrdqybagnz/3c3+aejliqick5p1fyuq9pm40xq9y7mvacref7kqi1p9vltev6ddgp78ddnovg7x3taj1d1pwhc8dqpht1x5r4ek3b+3cg80bu9ulvmdobp2sp117y9j+e8hjyirprfn8ewgda/8y9mruafyngfxz/yhx3sj/71copzt5o/n3ln7u9rf1nbxhut/w3t87vnaydrr9bctf/d8+7t3hv/4qcpxz30h15o6e/untr/xov9piz/dws7ovw=</latexit>

13 <latexit sha1_base64="pim2buniwj2/wyvvvvkikbweovg=">aaa0rxicnvtbc922ezz7tdvb0j72ha2kqrrlio7jpjlkpbpfvu1ohfw15dxesqosodyqedmanosz9lf0z/wv9km7am8hcycshm3yiobvf4vf7mjxuvaktmidg//e+8lpf/bzx/zynv+t//o3v/3d79997w9fi7zkix0v5knovw2ioanl6cvjzek/ltglazdqb4lrx9j+zzrywflsvc4kep6sognjfhijvzfv/mcrs4rrj596wc7eipyqbh3rzsw7qlzf79teiy/2ezqw24gu2htb5x3pf2v6wv09gtuxlz9azxg5rkcxrr1voj7vk+yby6vlt89zpje76yfb/psgsprxfsoitxcw9c66n+vzmqaur1++u3gwf6b+pptj1hxsrdu/j5fvfxjpj/kwtgkmw4qicty6kor5rbhkyulrdb8utcdhnynpgxxmjkxivflqq70tqim8cc7hxyy9vdulqegqxcinajksorfmg1a62s5kof7kvgjzuuqahbqjczl4mvdwlryicrh8soapenigsnowa5yeemzsvddnqrkysszvdawsdc7dpc36ttenxysf835twias8xzwrw3zkctimwf96nekfmtm9mspza/lrfkdmwyci7ewaqmz2i+m2mospuwnfx/xw3habxnuezmqdojnyjscbxocieecl6byrmmmcpvqtnnowz6d2pweoun7daimxiw2r4st3qviuutf9ofn6e5cwyonwi6soif5maqpjkehrqczihzgkjjokhewpzjala6iflycmkjumkml78poscknov9qwnc5zbgilxezmfmlrugzikervktymhlmir9nsvlbwhnkked9of4hfwmtshy+apgo+syros2jm5pnqzqmhapqpk+8lpmy1juf0jhllzpcdp8agg8ioabmewrmnkl8nxw0qksdmtbdbkuwyxh4thww/5hzctemjrw6qhtqn0srzidr2funbrizlqxev+ma70gj1pnxryzfmj76bhre+dp1ybqqickpjmrsaev4z97gynvb+/lk6xmjjrwxjii35hj8+plceozwcpifvtjuqzfwmpxno2ojszctrh3/7gzurbh15ohmvkl6fhvljzrl47o6ou6hvnpqiz/egefrpx4rvi9iwsbyyo0tqh/pgeom//j1cypenah0nfgo2qfs59hr/x/7ni6rjzhfqiwkwria7iwgv/jllc+wwglz73p+xkedcs/ncmhqk59ltkbyylbqq1pmfw4py8jcyb5dxc8nqu4qnvjlfgupi5baxpc15f1hmllgntmejttzk6vknuyrtfp0ewihayjrxf0jdoqbsckqeqgb50mei2geodcectfdbhffuopaf15u910i0fwhsmdciprbsqa4fo/gojobft87ymos4ydvwprwqgtd2fiiomyzw8xhegoep97x4ehj515bcso/tdumpgqrizfone9ygshnaqhs5rzi4pexzjq0lajxcoqqukz8hu/2wvvbm+8mqjgrbyzkiu3uqyqlzhfpgmfsyqlojishbmm4v07rohex9i1n+pvlxtdty6wjmb4nh+w+po7z/druy0ccegsw1lezearyrr54pf9a83h7vrvysdvygunwvibozmgzcmddtk8qh1qt0tqxpbveqjm8sfx4etso+lzae83pla32ejgj6wuhbhcu7rhx3ubgahpati++on5+sjttt2rnzkkmgggzrhtrnrdmzfpqqudosnfcigl3qpc0nxrnz2krtxywyacilcsctbaom5apgvwipc0gkkym6hckoax1xqmd9dpuefbttczpryewtqseznieqq6oasbmt1guswkoh6aosyeaadfpoavakwywrxiqc7kgeen8ybd7aqtwcdzmxmhx2fgxqrh0qqwuvlgw93kahwb5yekugaxodvmnjvtbj0vibld5syylewizxyqvnekog1z7o/0rts0kblcqo/8ffzjohbmaxkmger1ptlci+gl/y9ziot/mcytkys0ykkwja2gpd20hhr1oagk+hkjjxvlxbiobnecmtbya1xy1ar5ynupi3gsvcqozlrtmlnwk7uyqlrrw0wbedz2qpnpk2hlu5xobdzvtqlzsm69hagbrsb+diwwclegv2xqwsqtwq/w6kzcpliwwxebtw/7z794kgzhqe6lxhbu4vct2gpkk+tafuyuug91qrtatfcqkgiyt3eynekv8f5af7g+arju0lnquctsvulikoibewui+eh70cv2rubmqmi6mmep/u0gdxttrmojaf8bqerswsrdwl1gye+kcigbo0jh07txn6yqyn/yg5ledffcucikswamexnrnscyp4e0l+85ywcdi+auvsrqql9onhrgymy0b4sscjjldbxosjjmxfqozvvax1wevjsffdb00xt3v7orodmdfbq6iremoulizazzdd2i/cztga7qd8lvawvfw/3dzlawezwstg4wm8kzvplpmwqlygpktowbksyyz2vpg5syognkkbbrzm9vi+ljtyj4ngjntlo6bnw76orm87iquwz8oe6mbpuyqg+o6daygag+qetwopcup5n9g3hrdbta3vzd2kfd6tlhp11+aoq1aricageilnofxpt3rk/cva3xljgnpgjrhfbkr/w6lxekstpncibajsgyil9xngjq1czrx65ja3bjjrgm2vdecjqseslquhintb3m4dfgyxzjpx055agia7rgm7gomcs95k3zuexguql18vgqgc/11dqdwvd57cg4kmirxrbxxqhgpauy5c5emklpjjmat0xndh4oejjnkgy6lhspibmo4hzu9hsxagbpg9hikhsncjqh21utq25vxyj5uslpdchciprco8luavaaxnoirg0vugsfcaftqio5egayr1jbglkyn1jv4ky53mq48x1cavx30ndrq6obj0tqaweq6japrkrxbsqalm5qkhuqsd1xsbai2spkmktypkraaxszwtpbitds7svis7oz0evth6ys69gwnwowcomlnud/llwbanssaysnpajd0376m6cvh2qqs2gwtisg7qmfvxvl6qppaw8goqunim12zdmpigpdq1h2bb/6xsnc8b6qgbgwcim2ezpdqqha4kysrdsno7ubc6lufrfpayisaugadgcsq0ridsdx6hyfmwctc5qgsjc4iqn/ukms+2xjpfpncxcmtsreyqqj13nw5hcccuuzyzlqlbfmcgxrtoxbmhxypibwi2unijyb5e0lx1hy0vvdwunfrcirxcdqj6prre22aypsqofckygyyvgacrvhzarldfjbrkitxwsoyxfc3yoajv75ou8lnwpip1uwce9fzyky6ocmezckrjamsbq56yswfpznphoupkuts0pnc5cmniqoe3p2uk83bld7inqiddx2npxctuziloyzf2vfaenubeqzdrlgtqbcw7bzyl2qnnsfubbw+bmqf6zlqwt5rofd20xyz3j1m2owqe+tohg/jgsepv9uafk8jmeoxg8c+jqypun1a91pfczlj0vkgzmc4gh8xqx14yr9ic/szvttqlmt6udluqdac2p20xmtyzavlaoigsz9hnss2oiibqhdccjutwigqog2zdf76pbt8n9jb9zjnascwhnvqevrtwtk8yskuqcx6exhgbk13uicyli4djohwmyharw4elias2itflrci+nb7jfvcbugqerio1esqn2ai5bqznsece1yooojb44hzpuq3wyf/sq60ze1mnnu3ygu+xpfnar3cem6mehve9uwesiznvdkuczy1b0g2r5sbbuke+t6ngkww5/tmtebcihmokejr16tqmahndl3alg+i30dvt3buyocswecjnn8bwurutcoovutcml5lpcazluhvwdn+jhi9zrgjiiakhczu8zcsvqywrja28twzj0orxlnp3p9za/euzwkgazfqgompzrvu0xygzckvjtzq+25c4b1qzvfbhs5r1chrd8t80f/bladvftsmc6yxwkyezkoqoq1b11alk40l80r5qs45jnwpzmheqqvmgz7oz8znz3ccj27jxfgqumndwaysgbqfwqqaa/sf+ncpzb0xsyrfo/i+8fha5fxiiump6vl3kcb7cfu+rq+y33ijtrjlouanvfoxjdcv8+bqrgfl7gdh1r3rgyzzxrdmkhwaysdo8nppt9hgd8sljup9ihoydso+48yoagdndriawe4bysgviv/mzulva0sfii1zyfe/leycffukhcx6ep+jmk09+nnyjuyfdiwzdgp1lhctmu7dz9vrxfftt/ukuz0tplxhydc0qkecercwqdiv3a9c925++om38t/iyefwcnq1tnhb863axojbespzqhw4ht7mik7/kjw1u1nenm5sp2sbu/v2ttajscf3hl/r6mchn0/gmin7u1c3pipblmcbgiz+w18oj3qmrnadj3fgaxapcw52bixcs6rvviqthwdxqkyyuit5hbs+rxdxvf2c3cvhi6dh1aavgybxd3pxoqz4qizhlzjqouyrgjl++ffsatibo7hdyhuztcehgo1wkqtnlcr7ezxb2ftblk1f+isg0ek/gin8z4golcoelisr1zlsvqo0iouku26xqjsi/sk5fsji3mkxfdpnxpa4ptelxbkhbmgf4yzfphqhx0jwuysg1w+bc3xkwfc8dsa1d8f1dw/qpisnyq4eiivmawa+tjdrdqybagnz/3c3+aejliqick5p1fyuq9pm40xq9y7mvacref7kqi1p9vltev6ddgp78ddnovg7x3taj1d1pwhc8dqpht1x5r4ek3b+3cg80bu9ulvmdobp2sp117y9j+e8hjyirprfn8ewgda/8y9mruafyngfxz/yhx3sj/71copzt5o/n3ln7u9rf1nbxhut/w3t87vnaydrr9bctf/d8+7t3hv/4qcpxz30h15o6e/untr/xov9piz/dws7ovw=</latexit> <latexit sha1_base64="pw8oji3ujegpi9vjlgbxuppo7ly=">aaa0ohicnvvrct22ezbtw6rekuzn2xk2kqzylkk6jpnmkvfmffu1o3euxzzzsshrqbkhbxjvbsbzmcp+7tp2ifoo3qv4dkkaloroxhybfltylhyxi4ucimfc7u//99zbp/v5l375q7d/vf6b3/7u /fivykof0ezgnof8+iiimlkppjzmj/b7glkrbqr8llh9g+3dtygxlsxo5kohzsukmjvlijfsdv/ofrswprp986mu5e4twqrl1rdfn7gw1/ep27d33yj+hy7kd6krdevte8xxrpufvxf1r/blye+gsiucvnna19x3us3qffx1+sfz2oysn8vb68by/pagsfvxtdt/lszinkf8/f2djf29f/xj2x6j52fhrfo7p3/3wlh/lyznstiyjeej0tf/is4pwycke1ut+kwhbwkss01p4zehkxvmlnfz7w1ateeocw79meqq2s1grvihfggayjxiizdasdlwdlnl8yvnfsqkunat1r+my8wtuofq9iheyflkadxjybrj6ohroqgmttn7rpiajtfkw94zsstqxyz4w/dqyk2lcwnm/ni0tyxg+69eg+zrlscyzfvu4bb5ljvu1foahcikpwrkgzsvyglxrde0wru0xsvp2mn6w46044lcm97wzcdrxjmrkweg8acqgjhdqoam0u4r9dac5p2dcqw1ogpyk/tprbmmwm6pfce9v8vyiyvvau/qgoa2rrmbhszzd/ezsu8bqa6ptgrelmbucdmg4mb5mbryhrapbdhkkn5mhkvdhp6suk5wvkoe7hoyixmuidjipkqkgwctqcj5reza8imu/m1n5yk0jmhsgflcvkiuevlkox7s4x33m1dcw0rnnpibkhg5vndmnxpvmtorkd2jmskpnlnp8apegevaapgkfc5puvpo6wttvpk3rbomimleifkba3/vibiuakxiohdq9qe+isobdkoyqhgkmwutwr6txghekbor8yylgbxhpfto6q3ztojanvzcufec19rryvfnvy+tt7n55/ois0caks0beg3i5onh2zgldoaxjd55u6jhiomz65ugr0dnrcajepl4afvfepty8glwqhp9f0ypn0riudo96qk8eweinmubqwo9fhnvtkhawgljjhndhc+c46t97vckgn00afqu+inao1hn4au/fpqfjamnkssqlgjcupjulwa98gcuxwk2erhc8p+lttoxncwu0tzxpmicjbdbytvutsz63lgvhlmspoelneyf3fbv8couclbflbei7mvlombkpy5tsv5ptuslsmk/zicz9ugmbkade9y7+bab1wfhhvynqwpmkwsuwt9abgosylok4iilfrt9fvndccnbcb7jrqikawbkmulxwcfdndysxdvkw5dhittcllsrbcla8ii5zxjdzed6y56l3dhdw4ilxkilyt201gynbymimwqc4vmdierocwzmhopjinfmtlkxgckfdfeev3/ddxng/evpba1bkziehf2k36ygsbboinmewqeqenjlfeijhncunana7s/rgjvz6vgky6wxwhezfhen2b7pf7e5qqrejdgkcworbwtwb7fyppj4vohm8pd9z3lbb2ilp1r4gtmtbmqpnxbsvkoduldlumt073ki5ven14ek7dvi2wnvnt5st9hcyi+sfb2xwru0r8d7mxmhzwlqpvzp6crw07tdkz85jjhobmax00z63tmxauelntkjrqoojd0kqtj10z2dikbv8sgaaijqknlqqkjuqkyfciqqtbipmjoh3jdml9cu7buhv3clg22hn0o5olgsxpjzlpiichnnmzj+wkjyeud9auwzdnnbi0nakgfymlfrxvodydsls+znh99mw4bbszliq47cjllvj6ymykkywlo7zna8b8ozfkteynoimtkzotx+vkn2u8pimsyyemo5vkjvfhqetdkd7i5pasnhuvub/cyownqknnlyedpry05ygrphf/se4kun/zbmwlcjamcszegbchh3ydgx70qqsfayfxrqk4g1gnjmbggktnkr5ytv84wqgxlnjxhvgz1s2zfz4fnqnvfo04ccnvb96ih3mlluzqmr1nho221g52gdew9qmopn+bxplbcmrkrl1ygmxribqdvmuuyglyymn2rf8s9+9fqzxoczerzk2cklw7dqksfw9c6mwxqe71vptjvlifdamw7ingibw+f8gc9jfnf13ac3pkos2k6bksurnvlxqffhk8dn6v6nmvtrdtgpu/3adootp1jufgs6m1sm2lgrhrs+xzcbeouuankax6dy5z9ijzw7sfcwvbikuxbrewintcym1z5omu8cbfvad5bbokpjxk5v0ic+1twxxztzbnssxorgzhw40wexylvsuzqzsyy8dkkycl9xl0hr3vboro9mbf9c5ibinoedixg7ydn2h/chtga3qdsafamvfw/3f1vewejarta4wmscbvvhymgcnyglktusaks2xzgzrgzkbo2bkk7bpzi9ti+tjtsn7pgbktrs4bn4y6ido8larugz/muynbviwqw6q68sxgqo9qepwo/cipjj/gxltdllb3/xe20fe6tlhpv1/ayq2areeamagldocxpf2rk/afw7whtkmpgvohffkrvs7lxijstpncifajcozilxwn2no1m7pxk9kanfjjbsn2frdcpqsesdqurqvtl/l4tbeyx7lph0f5qkialjhmrkpmco9503auexhuqd18vgsgcz118unwph69o6zkcaoxi+rjbuke0wzylm4zisctsyxnozo6/4yjctjva50lpuyfjcbwzws7desbsbqmllhipuyfmomsk8mgl6vvalzpilvhzkbedg9dycv1egfjffexf0iohimfglygkj0j8jgilsgmajkpawuxftxuylx4pmc0qzqo6/rvkdxt47w0crs0nvujzlkfuoywfyljolqkrypkq4ttvgsnjxktpw01fqytmkcr2ra3kncynvnpi+gok4n1kvna8bib8bjpop6ww4ba6pz0qhleua7phv2zurfodymlcaumwwj2ruke+vk0hpsbnpcdkgdqprpmnvqqxkehkbuczzwj4w95jlqntrgeabpdmw+i0k1w11zixayrms34lrfkmydb1ze0yac68aguzxg7usq0u8nnmsifjzhlwb0qvd9q1zz8+ngskfiz+zkmzs9xvqvreouzu0afsxlknfwwiyyfxikanuxyo6wewdhqmgeqmoezntdire2opgfh8enxkmgjeoggvgnxjto0wsjh8y5vwazw6ay4ztcz0rl6lkwivzzuqpwksgguguzt/z1rztkbskst6ku4py4zgyz1ckp9mbowsfjkhrz1wmr/jzksedyt5vcl5lo5czpauwc8otmp1xpdm7xbrdbhbzraoyjm4islmlh/m6oaslb3swz6ckknusfsgy3xboko7yjpirg9xft+1zpgrl2xsobso/mm+xuemlov91br/n4ga8msbna1vr7ssbuuhg89jexfydq1rrxklqqsy6wmmq+xhh8ufjtwav7qjqjz6yigxyh04escqeceddupouixm4ulrnanez6ppjctter2ucf40aobbeouqvtm63z14fd4pvqdr5hou87sedivpfqbw+w4rvlpghi1ovam2jrvcmfxhrx6tbbhisjbg51i7aeycfellk8rcc33i/zf3zdv9gtrafegvg0eywnykg54dy1qsf+3x73zfzrv906+y91og1uz+lcvhur8pgkbflr5zzwy9tbilyvj1qyaeqvpoyzrhldsnnzdhat98x3obusgj2nz2ycehkzciguj7omvy9bbxlo0mz5e/0ger+2cwcdlwwgh6x53hbwd68drrva54lxnemxma1eegoe9ioryyndgbexusrkoz6hyfv5tws1ti1vmx0civtkeub+xxu4d2loa0mlhajmf2yucjvmacvlqwjkud83ewlvghkodzj0xqmowx9c5ov+j6y8eppbzphjvdriimqehcxp2lqgkllpljknyln1zmgsfnqoi1zdnc/wdd4zbjq7ethxs+bcgpaa6wo2zqhu+nrsbrzyl8chfizrjvkikrqv93eppuxojaaynpoudr9pvj+w79v6gpk1n2kla65jxts7h0sc8zpvrk50tsx+fmbda71hmtd1wyrspxkg4gyv/xslbxtxvna40c8yivs34jvojiog1myhqxnsvpgm8hlyzv4mvd2w9chskguw9vpjlthxr8broo/3pgrt2oofle9wxkdhommwvm8j0arppvxcvdz1p/1utzlvm2hkfrz2tamaxcebc5ukvdr9knt36qefebv/e558bi2nv8uevtxfqu0zukwnnavuh9vhgyzi9g/cuzov5c3pzhzkfri7b19rk4gu8vbwkxodoptmyvhzzg/36oz0mjjyoedqxk/rc+vrz9qvrumxdmza4nhowc6ghhtjtzqxsoxjia7ivazqma+q1vf1iq63x7b7oxj66lda9dudmo5h7lqufv+vjbwbezqharvq0h8/supnemuh10t5jzmpjesw0upy9axfex+czyubftuyqqgf0hba6fcwhv8x6fg57olcsnqdi2wlsonytrllfqtkg9gvksuhjmqtffmh3eqpiwmapculeuoq/blu+ox9efolbde1yld9tjlxp4x1xemaxhmhxhdnv+5d0ijm6igywuwd5mmle13gekiwkl0pd5p/ospckilgmlbn77o6at5obvxvam56silnrs7uklafnpvn1xgntgcpc56dtfnfb/xgvdedjhsvnvrye1xuqtaj55cdzfnd0qo1a24h+lctxhfn2wcyzmpii+qq+x0vdinh8xv/agzk/omm/fht3b3r/t7om3sbn3/s/pnm22t/wnt/bxtttpaptc/xhq8drz1fc9f+d+u9w3++9zd77997fo/re99o6fu3gpr31no/9178h/jgm6y=</latexit> Modeling for the Noisy Channel we need to model two probability distribudons: should favor fluent transladons should favor accurate/faithful transladons let s start with <latexit sha1_base64="pw8oji3ujegpi9vjlgbxuppo7ly=">aaa0ohicnvvrct22ezbtw6rekuzn2xk2kqzylkk6jpnmkvfmffu1o3euxzzzsshrqbkhbxjvbsbzmcp+7tp2ifoo3qv4dkkaloroxhybfltylhyxi4ucimfc7u//99zbp/v5l375q7d/vf6b3/7u /fivykof0ezgnof8+iiimlkppjzmj/b7glkrbqr8llh9g+3dtygxlsxo5kohzsukmjvlijfsdv/ofrswprp986mu5e4twqrl1rdfn7gw1/ep27d33yj+hy7kd6krdevte8xxrpufvxf1r/blye+gsiucvnna19x3us3qffx1+sfz2oysn8vb68by/pagsfvxtdt/lszinkf8/f2djf29f/xj2x6j52fhrfo7p3/3wlh/lyznstiyjeej0tf/is4pwycke1ut+kwhbwkss01p4zehkxvmlnfz7w1ateeocw79meqq2s1grvihfggayjxiizdasdlwdlnl8yvnfsqkunat1r+my8wtuofq9iheyflkadxjybrj6ohroqgmttn7rpiajtfkw94zsstqxyz4w/dqyk2lcwnm/ni0tyxg+69eg+zrlscyzfvu4bb5ljvu1foahcikpwrkgzsvyglxrde0wru0xsvp2mn6w46044lcm97wzcdrxjmrkweg8acqgjhdqoam0u4r9dac5p2dcqw1ogpyk/tprbmmwm6pfce9v8vyiyvvau/qgoa2rrmbhszzd/ezsu8bqa6ptgrelmbucdmg4mb5mbryhrapbdhkkn5mhkvdhp6suk5wvkoe7hoyixmuidjipkqkgwctqcj5reza8imu/m1n5yk0jmhsgflcvkiuevlkox7s4x33m1dcw0rnnpibkhg5vndmnxpvmtorkd2jmskpnlnp8apegevaapgkfc5puvpo6wttvpk3rbomimleifkba3/vibiuakxiohdq9qe+isobdkoyqhgkmwutwr6txghekbor8yylgbxhpfto6q3ztojanvzcufec19rryvfnvy+tt7n55/ois0caks0beg3i5onh2zgldoaxjd55u6jhiomz65ugr0dnrcajepl4afvfepty8glwqhp9f0ypn0riudo96qk8eweinmubqwo9fhnvtkhawgljjhndhc+c46t97vckgn00afqu+inao1hn4au/fpqfjamnkssqlgjcupjulwa98gcuxwk2erhc8p+lttoxncwu0tzxpmicjbdbytvutsz63lgvhlmspoelneyf3fbv8couclbflbei7mvlombkpy5tsv5ptuslsmk/zicz9ugmbkade9y7+bab1wfhhvynqwpmkwsuwt9abgosylok4iilfrt9fvndccnbcb7jrqikawbkmulxwcfdndysxdvkw5dhittcllsrbcla8ii5zxjdzed6y56l3dhdw4ilxkilyt201gynbymimwqc4vmdierocwzmhopjinfmtlkxgckfdfeev3/ddxng/evpba1bkziehf2k36ygsbboinmewqeqenjlfeijhncunana7s/rgjvz6vgky6wxwhezfhen2b7pf7e5qqrejdgkcworbwtwb7fyppj4vohm8pd9z3lbb2ilp1r4gtmtbmqpnxbsvkoduldlumt073ki5ven14ek7dvi2wnvnt5st9hcyi+sfb2xwru0r8d7mxmhzwlqpvzp6crw07tdkz85jjhobmax00z63tmxauelntkjrqoojd0kqtj10z2dikbv8sgaaijqknlqqkjuqkyfciqqtbipmjoh3jdml9cu7buhv3clg22hn0o5olgsxpjzlpiichnnmzj+wkjyeud9auwzdnnbi0nakgfymlfrxvodydsls+znh99mw4bbszliq47cjllvj6ymykkywlo7zna8b8ozfkteynoimtkzotx+vkn2u8pimsyyemo5vkjvfhqetdkd7i5pasnhuvub/cyownqknnlyedpry05ygrphf/se4kun/zbmwlcjamcszegbchh3ydgx70qqsfayfxrqk4g1gnjmbggktnkr5ytv84wqgxlnjxhvgz1s2zfz4fnqnvfo04ccnvb96ih3mlluzqmr1nho221g52gdew9qmopn+bxplbcmrkrl1ygmxribqdvmuuyglyymn2rf8s9+9fqzxoczerzk2cklw7dqksfw9c6mwxqe71vptjvlifdamw7ingibw+f8gc9jfnf13ac3pkos2k6bksurnvlxqffhk8dn6v6nmvtrdtgpu/3adootp1jufgs6m1sm2lgrhrs+xzcbeouuankax6dy5z9ijzw7sfcwvbikuxbrewintcym1z5omu8cbfvad5bbokpjxk5v0ic+1twxxztzbnssxorgzhw40wexylvsuzqzsyy8dkkycl9xl0hr3vboro9mbf9c5ibinoedixg7ydn2h/chtga3qdsafamvfw/3f1vewejarta4wmscbvvhymgcnyglktusaks2xzgzrgzkbo2bkk7bpzi9ti+tjtsn7pgbktrs4bn4y6ido8larugz/muynbviwqw6q68sxgqo9qepwo/cipjj/gxltdllb3/xe20fe6tlhpv1/ayq2areeamagldocxpf2rk/afw7whtkmpgvohffkrvs7lxijstpncifajcozilxwn2no1m7pxk9kanfjjbsn2frdcpqsesdqurqvtl/l4tbeyx7lph0f5qkialjhmrkpmco9503auexhuqd18vgsgcz118unwph69o6zkcaoxi+rjbuke0wzylm4zisctsyxnozo6/4yjctjva50lpuyfjcbwzws7desbsbqmllhipuyfmomsk8mgl6vvalzpilvhzkbedg9dycv1egfjffexf0iohimfglygkj0j8jgilsgmajkpawuxftxuylx4pmc0qzqo6/rvkdxt47w0crs0nvujzlkfuoywfyljolqkrypkq4ttvgsnjxktpw01fqytmkcr2ra3kncynvnpi+gok4n1kvna8bib8bjpop6ww4ba6pz0qhleua7phv2zurfodymlcaumwwj2ruke+vk0hpsbnpcdkgdqprpmnvqqxkehkbuczzwj4w95jlqntrgeabpdmw+i0k1w11zixayrms34lrfkmydb1ze0yac68aguzxg7usq0u8nnmsifjzhlwb0qvd9q1zz8+ngskfiz+zkmzs9xvqvreouzu0afsxlknfwwiyyfxikanuxyo6wewdhqmgeqmoezntdire2opgfh8enxkmgjeoggvgnxjto0wsjh8y5vwazw6ay4ztcz0rl6lkwivzzuqpwksgguguzt/z1rztkbskst6ku4py4zgyz1ckp9mbowsfjkhrz1wmr/jzksedyt5vcl5lo5czpauwc8otmp1xpdm7xbrdbhbzraoyjm4islmlh/m6oaslb3swz6ckknusfsgy3xboko7yjpirg9xft+1zpgrl2xsobso/mm+xuemlov91br/n4ga8msbna1vr7ssbuuhg89jexfydq1rrxklqqsy6wmmq+xhh8ufjtwav7qjqjz6yigxyh04escqeceddupouixm4ulrnanez6ppjctter2ucf40aobbeouqvtm63z14fd4pvqdr5hou87sedivpfqbw+w4rvlpghi1ovam2jrvcmfxhrx6tbbhisjbg51i7aeycfellk8rcc33i/zf3zdv9gtrafegvg0eywnykg54dy1qsf+3x73zfzrv906+y91og1uz+lcvhur8pgkbflr5zzwy9tbilyvj1qyaeqvpoyzrhldsnnzdhat98x3obusgj2nz2ycehkzciguj7omvy9bbxlo0mz5e/0ger+2cwcdlwwgh6x53hbwd68drrva54lxnemxma1eegoe9ioryyndgbexusrkoz6hyfv5tws1ti1vmx0civtkeub+xxu4d2loa0mlhajmf2yucjvmacvlqwjkud83ewlvghkodzj0xqmowx9c5ov+j6y8eppbzphjvdriimqehcxp2lqgkllpljknyln1zmgsfnqoi1zdnc/wdd4zbjq7ethxs+bcgpaa6wo2zqhu+nrsbrzyl8chfizrjvkikrqv93eppuxojaaynpoudr9pvj+w79v6gpk1n2kla65jxts7h0sc8zpvrk50tsx+fmbda71hmtd1wyrspxkg4gyv/xslbxtxvna40c8yivs34jvojiog1myhqxnsvpgm8hlyzv4mvd2w9chskguw9vpjlthxr8broo/3pgrt2oofle9wxkdhommwvm8j0arppvxcvdz1p/1utzlvm2hkfrz2tamaxcebc5ukvdr9knt36qefebv/e558bi2nv8uevtxfqu0zukwnnavuh9vhgyzi9g/cuzov5c3pzhzkfri7b19rk4gu8vbwkxodoptmyvhzzg/36oz0mjjyoedqxk/rc+vrz9qvrumxdmza4nhowc6ghhtjtzqxsoxjia7ivazqma+q1vf1iq63x7b7oxj66lda9dudmo5h7lqufv+vjbwbezqharvq0h8/supnemuh10t5jzmpjesw0upy9axfex+czyubftuyqqgf0hba6fcwhv8x6fg57olcsnqdi2wlsonytrllfqtkg9gvksuhjmqtffmh3eqpiwmapculeuoq/blu+ox9efolbde1yld9tjlxp4x1xemaxhmhxhdnv+5d0ijm6igywuwd5mmle13gekiwkl0pd5p/ospckilgmlbn77o6at5obvxvam56silnrs7uklafnpvn1xgntgcpc56dtfnfb/xgvdedjhsvnvrye1xuqtaj55cdzfnd0qo1a24h+lctxhfn2wcyzmpii+qq+x0vdinh8xv/agzk/omm/fht3b3r/t7om3sbn3/s/pnm22t/wnt/bxtttpaptc/xhq8drz1fc9f+d+u9w3++9zd77997fo/re99o6fu3gpr31no/9178h/jgm6y=</latexit> how do we compute the probability of an English sentence? language modeling is an important part of MT

14 Noisy Channel

15 Noisy Channel predicted transladon source sentence

16 Noisy Channel assumes we have the right model, and that we esdmate it perfectly

17 Noisy Channel assumes we have the right model, and that we esdmate it perfectly

18 Noisy Channel assumes we have the right model, and that we esdmate it perfectly extra parameters to tune, can tune to opdmize BLEU

19 Noisy Channel assumes we have the right model, and that we esdmate it perfectly extra parameters to tune, can tune to opdmize BLEU tuning

20 Noisy Channel à Linear Model? since we re not using idealized decoding rule anymore, why not add more feature funcdons? word count feature :

21 Noisy Channel à Linear Model? since we re not using idealized decoding rule anymore, why not add more feature funcdons? word count feature :

22 Noisy Channel à Linear Model? since we re not using idealized decoding rule anymore, why not add more feature funcdons? word count feature : reverse transladon model feature :

23 African National Congress opposition sanction Zimbabwe 非国大反对制裁津巴布韦

24 African National Congress opposition sanction Zimbabwe 非国大反对制裁津巴布韦 Gold standard: African NaDonal Congress opposes sancdons against Zimbabwe

25 African National Congress opposition sanction Zimbabwe 非国大反对制裁津巴布韦 Gold standard: African NaDonal Congress opposes sancdons against Zimbabwe BLEU score model score

26 African National Congress opposition sanction Zimbabwe 非国大反对制裁津巴布韦 Gold standard: African NaDonal Congress opposes sancdons against Zimbabwe BLEU score African NaDonal Congress opposidon sancdons against Zimbabwe African sancdoning to Zimbabwe s opposing predicted transladon opposidon to sancdons against Zimbabwe African NaDonal Congress model score

27 African National Congress opposition sanction Zimbabwe 非国大反对制裁津巴布韦 BLEU score Gold standard: African NaDonal Congress opposes sancdons against Zimbabwe learning moves transla1ons leb or right in this plot model score

28 African National Congress opposition sanction Zimbabwe 非国大反对制裁津巴布韦 Gold standard: African NaDonal Congress opposes sancdons against Zimbabwe BLEU score ideal model model score

29 African National Congress opposition sanction Zimbabwe 非国大反对制裁津巴布韦 Gold standard: African NaDonal Congress opposes sancdons against Zimbabwe BLEU score Where s the gold standard transla1on? model score

30 African National Congress opposition sanction Zimbabwe 非国大反对制裁津巴布韦 Gold standard: African NaDonal Congress opposes sancdons against Zimbabwe BLEU score Issue: gold standard transla1on is oben unreachable by the model Why? limited transla1on rules, free transla1ons, noisy data model score

31 Free TranslaDons Machine transla1on: Sharon's office said, leader of the main opposidon Labor Party has admifed defeat and congratulatory telephone calls to Sharon. Human- generated transla1on: According to a representadve of Sharon's office, the leader of the main opposidon Labor Party has admifed defeat and made the obligatory congratuladng telephone call to Sharon.

32 Free TranslaDons Even if gold standard transla1on was Machine reachable transla1on: by model, we might not Sharon's office said, leader of the main opposidon Labor Party has want admifed to defeat learn and from congratulatory it directly telephone calls to Sharon. Human- generated Applicable transla1on: to other tasks: According to a representadve of Sharon's office, the leader of the main opposidon summariza1on Labor Party has admifed defeat and made the obligatory congratuladng telephone call to Sharon. image cap1on genera1on

33 Loss FuncDons name loss where used cost ( 0-1 ) issue: gold standard transla1on is oben unreachable by the model intractable, but underlies direct error minimizadon perceptron hinge log perceptron algorithm (Rosenblaf, 1958) support vector machines, other large- margin algorithms logisdc regression, condidonal random fields, maximum entropy models 33

34 Loss FuncDons name loss where used cost ( 0-1 ) intractable, but it doesn t need to compute model score of gold standard! intractable, but underlies direct error minimizadon perceptron hinge log perceptron algorithm (Rosenblaf, 1958) support vector machines, other large- margin algorithms logisdc regression, condidonal random fields, maximum entropy models 34

35 MERT, Och (2003)

36 Minimum Error Rate Training (MERT) minimize the cost of the decoder output how bad are these transladons? e.g., negadve BLEU intractable in general how can we solve it? set of source sentences references decoder outputs

37 Minimum Error Rate Training (MERT) minimize the cost of the decoder output how bad are these transladons? e.g., negadve BLEU intractable in general how can we solve it? set of source sentences generate k- best lists of transladons, approximately references minimize cost decoder on k- best lists, outputs repeat with new parameters (pool k- best lists across iterates)

38 BLEU model score

39 BLEU each point is a transla1on for the same sentence Arabic- English, phrase- based model score

40 BLEU 10,000- best list, default Moses weights 1- best: 28 BLEU model score

41 BLEU same sentence, 10,000- best list aber MERT 1- best: 34 BLEU model score

42 BLEU another sentence, default Moses weights 1- best: 46 BLEU model score

43 BLEU same sentence, aber MERT 1- best: 62 BLEU model score

44 Why are there horizontal bands? BLEU model score

45 Why are there horizontal bands? BLEU latent deriva1ons, different transla1ons with same BLEU model score

46 Minimum Error Rate Training (MERT) What are some issues with this loss funcdon? DisconDnuous & non- convex opdmizadon relies on randomized search No regularizadon leads to overfinng As a result, MERT is only effecdve for very small models (<40 parameters)

47 Many researchers tried to improve MERT: Regulariza+on and Search for MERT (Cer et al., 2008) Random Restarts in MERT for MT (Moore & Quirk, 2008) Stabilizing MERT (Foster & Kuhn, 2009) Issues remain: Be;er Hypothesis Tes+ng for Sta+s+cal MT: Controlling for Op+mizer Instability (Clark et al., 2011) They suggest running MERT 3-5 Dmes due to its instability

48

49

50 Perceptron Loss BLEU score reference model score

51 Perceptron Loss BLEU score reference model predicdon model score

52 Perceptron Loss for MT? (Collins, 2002) BLEU score reference model predicdon model score

53 k- Best Perceptron for MT (Liang et al., 2006) BLEU score model predicdon model score

54 k- Best Perceptron for MT (Liang et al., 2006) BLEU score model predicdon model score

55 k- Best Perceptron for MT (Liang et al., 2006) BLEU score BLEU oracle on k- best list model predicdon model score

56 Ramp Loss MinimizaDon BLEU score model score

57 Ramp Loss MinimizaDon BLEU score model predicdon model score

58 Ramp Loss MinimizaDon BLEU score model predicdon fear transladon model score

59 Fear Ramp Loss (Do et al., 2008) BLEU score model predicdon fear transladon model score

60 Fear Ramp Loss (Do et al., 2008) BLEU score model predicdon gold standard fear transladon model score

61 Hope Ramp Loss (McAllester & Keshet, 2011; Liang et al., 2006) BLEU score model predicdon model score

62 Hope Ramp Loss (McAllester & Keshet, 2011; Liang et al., 2006) BLEU score hope transladon model predicdon model score

63 Hope- Fear Ramp Loss (Chiang et al., 2008; 2009; Cherry & Foster, 2012; Chiang, 2012) BLEU score hope transladon fear transladon model score

64 Hope- Fear Ramp Loss (Chiang et al., 2008; 2009; Cherry & Foster, 2012; Chiang, 2012) BLEU score hope transladon fear transladon model score

65 Experiments (Gimpel, 2012) averages over 8 test sets across 3 language pairs Moses Hiero (%BLEU) (%BLEU) MERT Fear Ramp (away from bad) Hope Ramp (toward good) Hope- Fear Ramp (toward good + away from bad)

66 Pairwise Ranking OpDmizaDon (Hopkins & May, 2011) BLEU score model score

67 Roadmap words, morphology, lexical semandcs text classificadon simple neural methods for NLP language modeling and word embeddings recurrent/recursive/convoludonal networks in NLP sequence labeling, HMMs, dynamic programming syntax and syntacdc parsing machine transladon semandcs 67

68 SemanDcs we talked a lot about lexical semandcs: word sense, WordNet, word embeddings/clusters how about semandcs beyond the word level?

69 ComposiDonal SemanDcs how should the meanings of words combine to create the meaning of something larger? there s currently a lot of work in producing vector representadons of sentences and documents simplest case: how should two word vectors be combined to create a vector for a bigram? lots of work in this area in the neural era, but earlier work began ~

70 EvaluaDng ComposiDonal SemanDcs compute similarity of two bigrams under your model, then compute correladon with human judgments: BigramSim BigramPara television programme tv set training programme educadon course bedroom window educadon officer (Mitchell and Lapata, 2010) 70

71 71

72 Bigram ComposiDon FuncDons 72

73 Bigram Similarity Results 73

74 Why does muldplicadon work? these vectors are built from co- occurrence counts (like in the first part of Assignment 1) so element- wise muldplicadon is like performing an AND operadon on context counts when using skip- gram word vectors (or other neural network- derived vectors), addidon oven works befer 74

75 75

76 Results SDS = simple distribudonal semandc NLM = neural language model 76

77 currently there is a lot of work on designing funcdonal architectures for bigram, phrase, and sentence similarity e.g., word averaging, recurrent neural networks, LSTMs, recursive neural networks, etc. our recent results find that, for sentence similarity, word averaging is a surprisingly strong baseline 77

78 on similar data to training data, LSTM does best 78