Machine Learning for Data Science (CS4786) Lecture11. Random Projections & Canonical Correlation Analysis

Transkrypt

1 Machine Learning for Data Science (CS4786) Lecture11 5 Random Projections & Canonical Correlation Analysis

2 The Tall, THE FAT AND THE UGLY n X d

3 The Tall, THE FAT AND THE UGLY d X > n X d n = n d d

4 The Tall, THE FAT AND THE UGLY d X > n X d n = n d d,k Eigs( d W = ) K

5 THE TALL, the Fat AND THE UGLY n X d

6 THE TALL, the Fat AND THE UGLY n X d SVD(X) d d V > n U n

7 THE TALL, the Fat AND THE UGLY n X d SVD(X) d K d V > n U n

8 THE TALL, THE FAT AND the Ugly X d and n so large we can t even store in memory Only have time to be linear in size(x) = n d I there any hope?

9 PICK A RANDOM W

10 PICK A RANDOM W Y = X d 7 5 p K K

11 WHY SHOULD RANDOM PROJECTIONS EVEN WORK?!

12 RANDOM PROJECTION What does it works even mean? Distances between all pairs of data-points in low dim. projection is roughly the same as their distances in the high dim. space. That is, when K is large enough, with high probability, for all pairs of data points i, j {1,...,n}, (1 ) y i y j 2 x i x j 2 (1 + ) y i y j 2

13 RANDOM PROJECTION What does it works even mean? Distances between all pairs of data-points in low dim. projection is roughly the same as their distances in the high dim. space. That is, when K is large enough, with high probability, for all pairs of data points i, j {1,...,n}, (1 ) y i y j 2 x i x j 2 (1 + ) y i y j 2

14 RANDOM PROJECTION What does it works even mean? Distances between all pairs of data-points in low dim. projection is roughly the same as their distances in the high dim. space. That is, when K is large enough, with high probability, for all pairs of data points i, j {1,...,n}, (1 ) y i y j 2 x i x j 2 (1 + ) y i y j 2

15 <latexit sha1_base64="dooojhwiqbghtwtykiinimx1knw=">aaacfhicdzdlsgmxfiyzxmu9vv26crzlrtokrwy7atgnywr2am0tmttthmyujbmhlpoobnwvny4ucevcnw9jpq2gogcsfr7/hjlzo6hgsip0ys0sli2vrcbwkusbm1vbqz3dugoisvmnbikqtycojrjpapprwzqhzmrzbgs4w/pyb9wwqxjgx+lrydoe6fvc5zrog7qp4wy2b03buvfdnlkvllfbyux+rrmg5ww8us7dcjevrjzccgmmy4elp8iiuqmyx0wiy8tugsyr2k29t3sbjtzmayqiui2mqt0ze6k5fwysbeekhyqosz+1jpsjx1rnpf1qag8n6ue3kob4gk7p94kx8zqaey7p9igeqn9edp/ywpf2i50x98nim5/ohnijaxua44rgj0tgtrgzqajk5q+qdogkvjsckyaer03h/6ketzgy8evjuni2jymb9sebyaimcqaclkav1aafd+abpifn6956tf6s11nrgjwf2qm/ynr7bhygm/m=</latexit> <latexit sha1_base64="dooojhwiqbghtwtykiinimx1knw=">aaacfhicdzdlsgmxfiyzxmu9vv26crzlrtokrwy7atgnywr2am0tmttthmyujbmhlpoobnwvny4ucevcnw9jpq2gogcsfr7/hjlzo6hgsip0ys0sli2vrcbwkusbm1vbqz3dugoisvmnbikqtycojrjpapprwzqhzmrzbgs4w/pyb9wwqxjgx+lrydoe6fvc5zrog7qp4wy2b03buvfdnlkvllfbyux+rrmg5ww8us7dcjevrjzccgmmy4elp8iiuqmyx0wiy8tugsyr2k29t3sbjtzmayqiui2mqt0ze6k5fwysbeekhyqosz+1jpsjx1rnpf1qag8n6ue3kob4gk7p94kx8zqaey7p9igeqn9edp/ywpf2i50x98nim5/ohnijaxua44rgj0tgtrgzqajk5q+qdogkvjsckyaer03h/6ketzgy8evjuni2jymb9sebyaimcqaclkav1aafd+abpifn6956tf6s11nrgjwf2qm/ynr7bhygm/m=</latexit> <latexit sha1_base64="dooojhwiqbghtwtykiinimx1knw=">aaacfhicdzdlsgmxfiyzxmu9vv26crzlrtokrwy7atgnywr2am0tmttthmyujbmhlpoobnwvny4ucevcnw9jpq2gogcsfr7/hjlzo6hgsip0ys0sli2vrcbwkusbm1vbqz3dugoisvmnbikqtycojrjpapprwzqhzmrzbgs4w/pyb9wwqxjgx+lrydoe6fvc5zrog7qp4wy2b03buvfdnlkvllfbyux+rrmg5ww8us7dcjevrjzccgmmy4elp8iiuqmyx0wiy8tugsyr2k29t3sbjtzmayqiui2mqt0ze6k5fwysbeekhyqosz+1jpsjx1rnpf1qag8n6ue3kob4gk7p94kx8zqaey7p9igeqn9edp/ywpf2i50x98nim5/ohnijaxua44rgj0tgtrgzqajk5q+qdogkvjsckyaer03h/6ketzgy8evjuni2jymb9sebyaimcqaclkav1aafd+abpifn6956tf6s11nrgjwf2qm/ynr7bhygm/m=</latexit> <latexit sha1_base64="dooojhwiqbghtwtykiinimx1knw=">aaacfhicdzdlsgmxfiyzxmu9vv26crzlrtokrwy7atgnywr2am0tmttthmyujbmhlpoobnwvny4ucevcnw9jpq2gogcsfr7/hjlzo6hgsip0ys0sli2vrcbwkusbm1vbqz3dugoisvmnbikqtycojrjpapprwzqhzmrzbgs4w/pyb9wwqxjgx+lrydoe6fvc5zrog7qp4wy2b03buvfdnlkvllfbyux+rrmg5ww8us7dcjevrjzccgmmy4elp8iiuqmyx0wiy8tugsyr2k29t3sbjtzmayqiui2mqt0ze6k5fwysbeekhyqosz+1jpsjx1rnpf1qag8n6ue3kob4gk7p94kx8zqaey7p9igeqn9edp/ywpf2i50x98nim5/ohnijaxua44rgj0tgtrgzqajk5q+qdogkvjsckyaer03h/6ketzgy8evjuni2jymb9sebyaimcqaclkav1aafd+abpifn6956tf6s11nrgjwf2qm/ynr7bhygm/m=</latexit> <latexit sha1_base64="ir6boziv9gol8fef+ofcpr4okwo=">aaacsnicdvbnbxmxfpsgam34suhi5ykiivnkv6hnd5uqupryjnjwyi4rr/o2sbr2ruy30gi1+xtcopxgj+dcayr6wdugt8filkyz8+znyapce+l8y9s7sxbz1u31jf6du/fupxhspjzyze0utlrzlo4kkx4lbxfcmgo8qrxkkxv4nj297pzjt+i8lu1rwlsyghlqda6vpccla7licfygnplmwd6ctym9iamsfip1c8vlemlcc3kmetdhh4bszah9fznqpltklqen7aw0lcsvazeohnzeordcqefezjypzhd3vcxgidorymhwoewhf/gsvlvbs6qq3k8fryhppcotcmz7ce2xkupmnui0ucsn+qs5qqkfp0gzqv66cczblfr7rcon9wuthws3vf/b68r/edoa8nhsafvvhfzdp5txbvajxa8w0w4vfytaphi67apqlp1ufnrvhxj+/bt+t462rokpxkvnw/0xqzrw2wp2hd1jgu2wfxbadtmekfaefwjf2nfoq/q5+hzdxkd70wrmefsdvbxvdzy0qa==</latexit> <latexit sha1_base64="ir6boziv9gol8fef+ofcpr4okwo=">aaacsnicdvbnbxmxfpsgam34suhi5ykiivnkv6hnd5uqupryjnjwyi4rr/o2sbr2ruy30gi1+xtcopxgj+dcayr6wdugt8filkyz8+znyapce+l8y9s7sxbz1u31jf6du/fupxhspjzyze0utlrzlo4kkx4lbxfcmgo8qrxkkxv4nj297pzjt+i8lu1rwlsyghlqda6vpccla7licfygnplmwd6ctym9iamsfip1c8vlemlcc3kmetdhh4bszah9fznqpltklqen7aw0lcsvazeohnzeordcqefezjypzhd3vcxgidorymhwoewhf/gsvlvbs6qq3k8fryhppcotcmz7ce2xkupmnui0ucsn+qs5qqkfp0gzqv66cczblfr7rcon9wuthws3vf/b68r/edoa8nhsafvvhfzdp5txbvajxa8w0w4vfytaphi67apqlp1ufnrvhxj+/bt+t462rokpxkvnw/0xqzrw2wp2hd1jgu2wfxbadtmekfaefwjf2nfoq/q5+hzdxkd70wrmefsdvbxvdzy0qa==</latexit> <latexit sha1_base64="ir6boziv9gol8fef+ofcpr4okwo=">aaacsnicdvbnbxmxfpsgam34suhi5ykiivnkv6hnd5uqupryjnjwyi4rr/o2sbr2ruy30gi1+xtcopxgj+dcayr6wdugt8filkyz8+znyapce+l8y9s7sxbz1u31jf6du/fupxhspjzyze0utlrzlo4kkx4lbxfcmgo8qrxkkxv4nj297pzjt+i8lu1rwlsyghlqda6vpccla7licfygnplmwd6ctym9iamsfip1c8vlemlcc3kmetdhh4bszah9fznqpltklqen7aw0lcsvazeohnzeordcqefezjypzhd3vcxgidorymhwoewhf/gsvlvbs6qq3k8fryhppcotcmz7ce2xkupmnui0ucsn+qs5qqkfp0gzqv66cczblfr7rcon9wuthws3vf/b68r/edoa8nhsafvvhfzdp5txbvajxa8w0w4vfytaphi67apqlp1ufnrvhxj+/bt+t462rokpxkvnw/0xqzrw2wp2hd1jgu2wfxbadtmekfaefwjf2nfoq/q5+hzdxkd70wrmefsdvbxvdzy0qa==</latexit> <latexit sha1_base64="ir6boziv9gol8fef+ofcpr4okwo=">aaacsnicdvbnbxmxfpsgam34suhi5ykiivnkv6hnd5uqupryjnjwyi4rr/o2sbr2ruy30gi1+xtcopxgj+dcayr6wdugt8filkyz8+znyapce+l8y9s7sxbz1u31jf6du/fupxhspjzyze0utlrzlo4kkx4lbxfcmgo8qrxkkxv4nj297pzjt+i8lu1rwlsyghlqda6vpccla7licfygnplmwd6ctym9iamsfip1c8vlemlcc3kmetdhh4bszah9fznqpltklqen7aw0lcsvazeohnzeordcqefezjypzhd3vcxgidorymhwoewhf/gsvlvbs6qq3k8fryhppcotcmz7ce2xkupmnui0ucsn+qs5qqkfp0gzqv66cczblfr7rcon9wuthws3vf/b68r/edoa8nhsafvvhfzdp5txbvajxa8w0w4vfytaphi67apqlp1ufnrvhxj+/bt+t462rokpxkvnw/0xqzrw2wp2hd1jgu2wfxbadtmekfaefwjf2nfoq/q5+hzdxkd70wrmefsdvbxvdzy0qa==</latexit> WHY SHOULD RANDOM PROJECTIONS EVEN WORK?! Lets = start with a one dimensional projection = (K = 1) y t = x > t u where each u[i] = random ± 1 = [ ] [ ] What is the expected value of: 1. y t y s? 2. (y t y s ) 2?

16 = WHY SHOULD RANDOM PROJECTIONS EVEN WORK?! = = = Hence= for any s, t {1,...,n}, = = [ ] [ ] Lets try this in Matlab... Ey s y t 2 = x s x t 2 2

17 = WHY SHOULD RANDOM PROJECTIONS EVEN WORK?! = = = Hence= for any s, t {1,...,n}, = = [ ] [ ] Lets try this in Matlab... Ey s y t 2 = x s x t 2 2 Law of large numbers says that average over multiple draws is close to expectation

18 PICK A RANDOM W

19 PICK A RANDOM W Y = X d 7 5 p K K

20 <latexit sha1_base64="piwkdkn7kx4tj0ajldd+otvtvhq=">aaacy3icdvfba9rafj5erxwrnq3erhi4cj7wtcl2exckxgqvfdy2kmqwmx3pdptj0pkx7rqyp9kbny/+dyfbrvtrbwmf3/e9n2++yepcwqrd755/6/adjbub9wzb9x883a52dk9s1rije1kvltnlhmvcltghrqwe1qafzgo8zezve/30mxqrqvijlwpmtdgvva6kieelwddyc5plebvouormcbyga+bsmz9iqurftnolc3gz2wtd7ftuuvxcthhjrrdfzmgqumgzddfdkvpnxlvaihja6q7iwgnpg2e4csoqcw494aevqgcod8d7fay8l1wn2bqo0+bbpk1ko7ekwqhrix7wlltckjifdoo4svglorfngdlyco02avczdfdcmvpik+nosbbib3a0qlu70jlz9vvbv7we/jcwnzspk1avdunyyqul8qyaqqaphkbkokri4ycqrrldqc6eezlctwxccncvhf+dk70rd0f8w/7w6m06jk32hd1jlxhnb+yivwphbmik++ftente4p30t/xd//gv1ffwpy/yh+u//qvm1bm9</latexit> <latexit sha1_base64="piwkdkn7kx4tj0ajldd+otvtvhq=">aaacy3icdvfba9rafj5erxwrnq3erhi4cj7wtcl2exckxgqvfdy2kmqwmx3pdptj0pkx7rqyp9kbny/+dyfbrvtrbwmf3/e9n2++yepcwqrd755/6/adjbub9wzb9x883a52dk9s1rije1kvltnlhmvcltghrqwe1qafzgo8zezve/30mxqrqvijlwpmtdgvva6kieelwddyc5plebvouormcbyga+bsmz9iqurftnolc3gz2wtd7ftuuvxcthhjrrdfzmgqumgzddfdkvpnxlvaihja6q7iwgnpg2e4csoqcw494aevqgcod8d7fay8l1wn2bqo0+bbpk1ko7ekwqhrix7wlltckjifdoo4svglorfngdlyco02avczdfdcmvpik+nosbbib3a0qlu70jlz9vvbv7we/jcwnzspk1avdunyyqul8qyaqqaphkbkokri4ycqrrldqc6eezlctwxccncvhf+dk70rd0f8w/7w6m06jk32hd1jlxhnb+yivwphbmik++ftente4p30t/xd//gv1ffwpy/yh+u//qvm1bm9</latexit> <latexit sha1_base64="piwkdkn7kx4tj0ajldd+otvtvhq=">aaacy3icdvfba9rafj5erxwrnq3erhi4cj7wtcl2exckxgqvfdy2kmqwmx3pdptj0pkx7rqyp9kbny/+dyfbrvtrbwmf3/e9n2++yepcwqrd755/6/adjbub9wzb9x883a52dk9s1rije1kvltnlhmvcltghrqwe1qafzgo8zezve/30mxqrqvijlwpmtdgvva6kieelwddyc5plebvouormcbyga+bsmz9iqurftnolc3gz2wtd7ftuuvxcthhjrrdfzmgqumgzddfdkvpnxlvaihja6q7iwgnpg2e4csoqcw494aevqgcod8d7fay8l1wn2bqo0+bbpk1ko7ekwqhrix7wlltckjifdoo4svglorfngdlyco02avczdfdcmvpik+nosbbib3a0qlu70jlz9vvbv7we/jcwnzspk1avdunyyqul8qyaqqaphkbkokri4ycqrrldqc6eezlctwxccncvhf+dk70rd0f8w/7w6m06jk32hd1jlxhnb+yivwphbmik++ftente4p30t/xd//gv1ffwpy/yh+u//qvm1bm9</latexit> <latexit sha1_base64="piwkdkn7kx4tj0ajldd+otvtvhq=">aaacy3icdvfba9rafj5erxwrnq3erhi4cj7wtcl2exckxgqvfdy2kmqwmx3pdptj0pkx7rqyp9kbny/+dyfbrvtrbwmf3/e9n2++yepcwqrd755/6/adjbub9wzb9x883a52dk9s1rije1kvltnlhmvcltghrqwe1qafzgo8zezve/30mxqrqvijlwpmtdgvva6kieelwddyc5plebvouormcbyga+bsmz9iqurftnolc3gz2wtd7ftuuvxcthhjrrdfzmgqumgzddfdkvpnxlvaihja6q7iwgnpg2e4csoqcw494aevqgcod8d7fay8l1wn2bqo0+bbpk1ko7ekwqhrix7wlltckjifdoo4svglorfngdlyco02avczdfdcmvpik+nosbbib3a0qlu70jlz9vvbv7we/jcwnzspk1avdunyyqul8qyaqqaphkbkokri4ycqrrldqc6eezlctwxccncvhf+dk70rd0f8w/7w6m06jk32hd1jlxhnb+yivwphbmik++ftente4p30t/xd//gv1ffwpy/yh+u//qvm1bm9</latexit> WHY SHOULD RANDOM PROJECTIONS EVEN WORK?! Like = repeating the experiment K times and = averaging y t [k] =x > t u k / p K where each u k [i] = random ± 1 = [ ] [ ]

21 <latexit sha1_base64="piwkdkn7kx4tj0ajldd+otvtvhq=">aaacy3icdvfba9rafj5erxwrnq3erhi4cj7wtcl2exckxgqvfdy2kmqwmx3pdptj0pkx7rqyp9kbny/+dyfbrvtrbwmf3/e9n2++yepcwqrd755/6/adjbub9wzb9x883a52dk9s1rije1kvltnlhmvcltghrqwe1qafzgo8zezve/30mxqrqvijlwpmtdgvva6kieelwddyc5plebvouormcbyga+bsmz9iqurftnolc3gz2wtd7ftuuvxcthhjrrdfzmgqumgzddfdkvpnxlvaihja6q7iwgnpg2e4csoqcw494aevqgcod8d7fay8l1wn2bqo0+bbpk1ko7ekwqhrix7wlltckjifdoo4svglorfngdlyco02avczdfdcmvpik+nosbbib3a0qlu70jlz9vvbv7we/jcwnzspk1avdunyyqul8qyaqqaphkbkokri4ycqrrldqc6eezlctwxccncvhf+dk70rd0f8w/7w6m06jk32hd1jlxhnb+yivwphbmik++ftente4p30t/xd//gv1ffwpy/yh+u//qvm1bm9</latexit> <latexit sha1_base64="piwkdkn7kx4tj0ajldd+otvtvhq=">aaacy3icdvfba9rafj5erxwrnq3erhi4cj7wtcl2exckxgqvfdy2kmqwmx3pdptj0pkx7rqyp9kbny/+dyfbrvtrbwmf3/e9n2++yepcwqrd755/6/adjbub9wzb9x883a52dk9s1rije1kvltnlhmvcltghrqwe1qafzgo8zezve/30mxqrqvijlwpmtdgvva6kieelwddyc5plebvouormcbyga+bsmz9iqurftnolc3gz2wtd7ftuuvxcthhjrrdfzmgqumgzddfdkvpnxlvaihja6q7iwgnpg2e4csoqcw494aevqgcod8d7fay8l1wn2bqo0+bbpk1ko7ekwqhrix7wlltckjifdoo4svglorfngdlyco02avczdfdcmvpik+nosbbib3a0qlu70jlz9vvbv7we/jcwnzspk1avdunyyqul8qyaqqaphkbkokri4ycqrrldqc6eezlctwxccncvhf+dk70rd0f8w/7w6m06jk32hd1jlxhnb+yivwphbmik++ftente4p30t/xd//gv1ffwpy/yh+u//qvm1bm9</latexit> <latexit sha1_base64="piwkdkn7kx4tj0ajldd+otvtvhq=">aaacy3icdvfba9rafj5erxwrnq3erhi4cj7wtcl2exckxgqvfdy2kmqwmx3pdptj0pkx7rqyp9kbny/+dyfbrvtrbwmf3/e9n2++yepcwqrd755/6/adjbub9wzb9x883a52dk9s1rije1kvltnlhmvcltghrqwe1qafzgo8zezve/30mxqrqvijlwpmtdgvva6kieelwddyc5plebvouormcbyga+bsmz9iqurftnolc3gz2wtd7ftuuvxcthhjrrdfzmgqumgzddfdkvpnxlvaihja6q7iwgnpg2e4csoqcw494aevqgcod8d7fay8l1wn2bqo0+bbpk1ko7ekwqhrix7wlltckjifdoo4svglorfngdlyco02avczdfdcmvpik+nosbbib3a0qlu70jlz9vvbv7we/jcwnzspk1avdunyyqul8qyaqqaphkbkokri4ycqrrldqc6eezlctwxccncvhf+dk70rd0f8w/7w6m06jk32hd1jlxhnb+yivwphbmik++ftente4p30t/xd//gv1ffwpy/yh+u//qvm1bm9</latexit> <latexit sha1_base64="piwkdkn7kx4tj0ajldd+otvtvhq=">aaacy3icdvfba9rafj5erxwrnq3erhi4cj7wtcl2exckxgqvfdy2kmqwmx3pdptj0pkx7rqyp9kbny/+dyfbrvtrbwmf3/e9n2++yepcwqrd755/6/adjbub9wzb9x883a52dk9s1rije1kvltnlhmvcltghrqwe1qafzgo8zezve/30mxqrqvijlwpmtdgvva6kieelwddyc5plebvouormcbyga+bsmz9iqurftnolc3gz2wtd7ftuuvxcthhjrrdfzmgqumgzddfdkvpnxlvaihja6q7iwgnpg2e4csoqcw494aevqgcod8d7fay8l1wn2bqo0+bbpk1ko7ekwqhrix7wlltckjifdoo4svglorfngdlyco02avczdfdcmvpik+nosbbib3a0qlu70jlz9vvbv7we/jcwnzspk1avdunyyqul8qyaqqaphkbkokri4ycqrrldqc6eezlctwxccncvhf+dk70rd0f8w/7w6m06jk32hd1jlxhnb+yivwphbmik++ftente4p30t/xd//gv1ffwpy/yh+u//qvm1bm9</latexit> <latexit sha1_base64="y6qd7ctg3gwq+5slaxnajaose5o=">aaadj3icrvllbhmxfpumrxjekszzwesvyojgr4imi0gvbjcykrjpk8wtkcfxjnz4hrlvvi2m8zds+bu2siaqlpktpe2oeh4sc65k6eicc30fdlrozygqb55/5eq16ze2brzu3b5z9157+/6rzusj5ejkojcnebdsq0yoqigwj4wrpi20pi6sl41+fcqnvxn2bhafdfi+y1ssbadhhdveyjelhozrxc3q0i6tad/baww45vgkhweyarndpfnmsrmgexfpluswwct1bvu7grk1m4o78omqsdyoo18vtvnzsvowt6xsyzsskggtj0p8z+3ghouk1tww3sj/v2oe7q7pekiopbgbdo85cwb/v9+jfuwbyuuhreiwbh9k01yuqcxaag7tmjicgoobuellusvkkwsuej6tywcznkobvbfvxomdx0xxnbt3msax7hpgxvnrf2nkne2z9letif+kjuui+0glsqiemyllobjughlcfbo8vuyk0ashuddk9yrfnlsfg/talbeen5piv4ojxpesln39rhpwyrwolfqqpuk7iki9dibeoum0qsj76733pnmf/xf+b/+l/3vp9b1vzgo0ef73h7xcbw0=</latexit> <latexit sha1_base64="y6qd7ctg3gwq+5slaxnajaose5o=">aaadj3icrvllbhmxfpumrxjekszzwesvyojgr4imi0gvbjcykrjpk8wtkcfxjnz4hrlvvi2m8zds+bu2siaqlpktpe2oeh4sc65k6eicc30fdlrozygqb55/5eq16ze2brzu3b5z9157+/6rzusj5ejkojcnebdsq0yoqigwj4wrpi20pi6sl41+fcqnvxn2bhafdfi+y1ssbadhhdveyjelhozrxc3q0i6tad/baww45vgkhweyarndpfnmsrmgexfpluswwct1bvu7grk1m4o78omqsdyoo18vtvnzsvowt6xsyzsskggtj0p8z+3ghouk1tww3sj/v2oe7q7pekiopbgbdo85cwb/v9+jfuwbyuuhreiwbh9k01yuqcxaag7tmjicgoobuellusvkkwsuej6tywcznkobvbfvxomdx0xxnbt3msax7hpgxvnrf2nkne2z9letif+kjuui+0glsqiemyllobjughlcfbo8vuyk0ashuddk9yrfnlsfg/talbeen5piv4ojxpesln39rhpwyrwolfqqpuk7iki9dibeoum0qsj76733pnmf/xf+b/+l/3vp9b1vzgo0ef73h7xcbw0=</latexit> <latexit sha1_base64="y6qd7ctg3gwq+5slaxnajaose5o=">aaadj3icrvllbhmxfpumrxjekszzwesvyojgr4imi0gvbjcykrjpk8wtkcfxjnz4hrlvvi2m8zds+bu2siaqlpktpe2oeh4sc65k6eicc30fdlrozygqb55/5eq16ze2brzu3b5z9157+/6rzusj5ejkojcnebdsq0yoqigwj4wrpi20pi6sl41+fcqnvxn2bhafdfi+y1ssbadhhdveyjelhozrxc3q0i6tad/baww45vgkhweyarndpfnmsrmgexfpluswwct1bvu7grk1m4o78omqsdyoo18vtvnzsvowt6xsyzsskggtj0p8z+3ghouk1tww3sj/v2oe7q7pekiopbgbdo85cwb/v9+jfuwbyuuhreiwbh9k01yuqcxaag7tmjicgoobuellusvkkwsuej6tywcznkobvbfvxomdx0xxnbt3msax7hpgxvnrf2nkne2z9letif+kjuui+0glsqiemyllobjughlcfbo8vuyk0ashuddk9yrfnlsfg/talbeen5piv4ojxpesln39rhpwyrwolfqqpuk7iki9dibeoum0qsj76733pnmf/xf+b/+l/3vp9b1vzgo0ef73h7xcbw0=</latexit> <latexit sha1_base64="y6qd7ctg3gwq+5slaxnajaose5o=">aaadj3icrvllbhmxfpumrxjekszzwesvyojgr4imi0gvbjcykrjpk8wtkcfxjnz4hrlvvi2m8zds+bu2siaqlpktpe2oeh4sc65k6eicc30fdlrozygqb55/5eq16ze2brzu3b5z9157+/6rzusj5ejkojcnebdsq0yoqigwj4wrpi20pi6sl41+fcqnvxn2bhafdfi+y1ssbadhhdveyjelhozrxc3q0i6tad/baww45vgkhweyarndpfnmsrmgexfpluswwct1bvu7grk1m4o78omqsdyoo18vtvnzsvowt6xsyzsskggtj0p8z+3ghouk1tww3sj/v2oe7q7pekiopbgbdo85cwb/v9+jfuwbyuuhreiwbh9k01yuqcxaag7tmjicgoobuellusvkkwsuej6tywcznkobvbfvxomdx0xxnbt3msax7hpgxvnrf2nkne2z9letif+kjuui+0glsqiemyllobjughlcfbo8vuyk0ashuddk9yrfnlsfg/talbeen5piv4ojxpesln39rhpwyrwolfqqpuk7iki9dibeoum0qsj76733pnmf/xf+b/+l/3vp9b1vzgo0ef73h7xcbw0=</latexit> WHY SHOULD RANDOM PROJECTIONS EVEN WORK?! Like = repeating the experiment K times and = averaging y t [k] =x > t u k / p K where each u k [i] = random ± 1 = [ ] [ ] (y s [k] y t [k]) 2 = x > t u k x > s u k 2 /K

22 <latexit sha1_base64="3yp9ejgszib0dby1vx1w49yzwpo=">aaadj3icrvllbhmxfpumrxjekszzwesvyojgr4imi0gvbjcykrjpk8wtkcfxjnz4hrlvvi2m8zds+bu2siaqlpktpe2oeh4sc65k6eicc30fdlrozygqb55/5eq16ze2brzu3b5z9157+/6rzusj5ejkojcnebdsq0yoqigwj4wrpi20pi6sl41+fcqnvxn2bhafdfi+y1ssbadhhdveygexprzmuvwt6tcokwa/wwsmoobxpichmgkzw6x5zektbnlxas7rmfnldqb7u4eznzudu/dpklewo1+vtfnysvowtyxsyzsskggtj0p8z93ghouk1tww3sj/v1oe7q7pekiopbgbdo85cwb/v9+jfuwbyuuhreiwbh9k01yuqcxaag7tmjicgoobuellusvkkwsuej6tywcznkobvbfvxomdx0xxnbt3msax7hpgxvnrf2nkne2z9letif+kjuui+0glsqiemyllobjughlcfbo8vuyk0ashuddk9yrfnlsfg/talbeen5piv4ojxpesln39rhpwyrwolfqqpuk7iki9dibeoum0qsj76733pnmf/xf+b/+l/3vp9b1vzgo0ef73h6bfbw0=</latexit> <latexit sha1_base64="3yp9ejgszib0dby1vx1w49yzwpo=">aaadj3icrvllbhmxfpumrxjekszzwesvyojgr4imi0gvbjcykrjpk8wtkcfxjnz4hrlvvi2m8zds+bu2siaqlpktpe2oeh4sc65k6eicc30fdlrozygqb55/5eq16ze2brzu3b5z9157+/6rzusj5ejkojcnebdsq0yoqigwj4wrpi20pi6sl41+fcqnvxn2bhafdfi+y1ssbadhhdveygexprzmuvwt6tcokwa/wwsmoobxpichmgkzw6x5zektbnlxas7rmfnldqb7u4eznzudu/dpklewo1+vtfnysvowtyxsyzsskggtj0p8z93ghouk1tww3sj/v1oe7q7pekiopbgbdo85cwb/v9+jfuwbyuuhreiwbh9k01yuqcxaag7tmjicgoobuellusvkkwsuej6tywcznkobvbfvxomdx0xxnbt3msax7hpgxvnrf2nkne2z9letif+kjuui+0glsqiemyllobjughlcfbo8vuyk0ashuddk9yrfnlsfg/talbeen5piv4ojxpesln39rhpwyrwolfqqpuk7iki9dibeoum0qsj76733pnmf/xf+b/+l/3vp9b1vzgo0ef73h6bfbw0=</latexit> <latexit sha1_base64="3yp9ejgszib0dby1vx1w49yzwpo=">aaadj3icrvllbhmxfpumrxjekszzwesvyojgr4imi0gvbjcykrjpk8wtkcfxjnz4hrlvvi2m8zds+bu2siaqlpktpe2oeh4sc65k6eicc30fdlrozygqb55/5eq16ze2brzu3b5z9157+/6rzusj5ejkojcnebdsq0yoqigwj4wrpi20pi6sl41+fcqnvxn2bhafdfi+y1ssbadhhdveygexprzmuvwt6tcokwa/wwsmoobxpichmgkzw6x5zektbnlxas7rmfnldqb7u4eznzudu/dpklewo1+vtfnysvowtyxsyzsskggtj0p8z93ghouk1tww3sj/v1oe7q7pekiopbgbdo85cwb/v9+jfuwbyuuhreiwbh9k01yuqcxaag7tmjicgoobuellusvkkwsuej6tywcznkobvbfvxomdx0xxnbt3msax7hpgxvnrf2nkne2z9letif+kjuui+0glsqiemyllobjughlcfbo8vuyk0ashuddk9yrfnlsfg/talbeen5piv4ojxpesln39rhpwyrwolfqqpuk7iki9dibeoum0qsj76733pnmf/xf+b/+l/3vp9b1vzgo0ef73h6bfbw0=</latexit> <latexit sha1_base64="3yp9ejgszib0dby1vx1w49yzwpo=">aaadj3icrvllbhmxfpumrxjekszzwesvyojgr4imi0gvbjcykrjpk8wtkcfxjnz4hrlvvi2m8zds+bu2siaqlpktpe2oeh4sc65k6eicc30fdlrozygqb55/5eq16ze2brzu3b5z9157+/6rzusj5ejkojcnebdsq0yoqigwj4wrpi20pi6sl41+fcqnvxn2bhafdfi+y1ssbadhhdveygexprzmuvwt6tcokwa/wwsmoobxpichmgkzw6x5zektbnlxas7rmfnldqb7u4eznzudu/dpklewo1+vtfnysvowtyxsyzsskggtj0p8z93ghouk1tww3sj/v1oe7q7pekiopbgbdo85cwb/v9+jfuwbyuuhreiwbh9k01yuqcxaag7tmjicgoobuellusvkkwsuej6tywcznkobvbfvxomdx0xxnbt3msax7hpgxvnrf2nkne2z9letif+kjuui+0glsqiemyllobjughlcfbo8vuyk0ashuddk9yrfnlsfg/talbeen5piv4ojxpesln39rhpwyrwolfqqpuk7iki9dibeoum0qsj76733pnmf/xf+b/+l/3vp9b1vzgo0ef73h6bfbw0=</latexit> <latexit sha1_base64="piwkdkn7kx4tj0ajldd+otvtvhq=">aaacy3icdvfba9rafj5erxwrnq3erhi4cj7wtcl2exckxgqvfdy2kmqwmx3pdptj0pkx7rqyp9kbny/+dyfbrvtrbwmf3/e9n2++yepcwqrd755/6/adjbub9wzb9x883a52dk9s1rije1kvltnlhmvcltghrqwe1qafzgo8zezve/30mxqrqvijlwpmtdgvva6kieelwddyc5plebvouormcbyga+bsmz9iqurftnolc3gz2wtd7ftuuvxcthhjrrdfzmgqumgzddfdkvpnxlvaihja6q7iwgnpg2e4csoqcw494aevqgcod8d7fay8l1wn2bqo0+bbpk1ko7ekwqhrix7wlltckjifdoo4svglorfngdlyco02avczdfdcmvpik+nosbbib3a0qlu70jlz9vvbv7we/jcwnzspk1avdunyyqul8qyaqqaphkbkokri4ycqrrldqc6eezlctwxccncvhf+dk70rd0f8w/7w6m06jk32hd1jlxhnb+yivwphbmik++ftente4p30t/xd//gv1ffwpy/yh+u//qvm1bm9</latexit> <latexit sha1_base64="piwkdkn7kx4tj0ajldd+otvtvhq=">aaacy3icdvfba9rafj5erxwrnq3erhi4cj7wtcl2exckxgqvfdy2kmqwmx3pdptj0pkx7rqyp9kbny/+dyfbrvtrbwmf3/e9n2++yepcwqrd755/6/adjbub9wzb9x883a52dk9s1rije1kvltnlhmvcltghrqwe1qafzgo8zezve/30mxqrqvijlwpmtdgvva6kieelwddyc5plebvouormcbyga+bsmz9iqurftnolc3gz2wtd7ftuuvxcthhjrrdfzmgqumgzddfdkvpnxlvaihja6q7iwgnpg2e4csoqcw494aevqgcod8d7fay8l1wn2bqo0+bbpk1ko7ekwqhrix7wlltckjifdoo4svglorfngdlyco02avczdfdcmvpik+nosbbib3a0qlu70jlz9vvbv7we/jcwnzspk1avdunyyqul8qyaqqaphkbkokri4ycqrrldqc6eezlctwxccncvhf+dk70rd0f8w/7w6m06jk32hd1jlxhnb+yivwphbmik++ftente4p30t/xd//gv1ffwpy/yh+u//qvm1bm9</latexit> <latexit sha1_base64="piwkdkn7kx4tj0ajldd+otvtvhq=">aaacy3icdvfba9rafj5erxwrnq3erhi4cj7wtcl2exckxgqvfdy2kmqwmx3pdptj0pkx7rqyp9kbny/+dyfbrvtrbwmf3/e9n2++yepcwqrd755/6/adjbub9wzb9x883a52dk9s1rije1kvltnlhmvcltghrqwe1qafzgo8zezve/30mxqrqvijlwpmtdgvva6kieelwddyc5plebvouormcbyga+bsmz9iqurftnolc3gz2wtd7ftuuvxcthhjrrdfzmgqumgzddfdkvpnxlvaihja6q7iwgnpg2e4csoqcw494aevqgcod8d7fay8l1wn2bqo0+bbpk1ko7ekwqhrix7wlltckjifdoo4svglorfngdlyco02avczdfdcmvpik+nosbbib3a0qlu70jlz9vvbv7we/jcwnzspk1avdunyyqul8qyaqqaphkbkokri4ycqrrldqc6eezlctwxccncvhf+dk70rd0f8w/7w6m06jk32hd1jlxhnb+yivwphbmik++ftente4p30t/xd//gv1ffwpy/yh+u//qvm1bm9</latexit> <latexit sha1_base64="piwkdkn7kx4tj0ajldd+otvtvhq=">aaacy3icdvfba9rafj5erxwrnq3erhi4cj7wtcl2exckxgqvfdy2kmqwmx3pdptj0pkx7rqyp9kbny/+dyfbrvtrbwmf3/e9n2++yepcwqrd755/6/adjbub9wzb9x883a52dk9s1rije1kvltnlhmvcltghrqwe1qafzgo8zezve/30mxqrqvijlwpmtdgvva6kieelwddyc5plebvouormcbyga+bsmz9iqurftnolc3gz2wtd7ftuuvxcthhjrrdfzmgqumgzddfdkvpnxlvaihja6q7iwgnpg2e4csoqcw494aevqgcod8d7fay8l1wn2bqo0+bbpk1ko7ekwqhrix7wlltckjifdoo4svglorfngdlyco02avczdfdcmvpik+nosbbib3a0qlu70jlz9vvbv7we/jcwnzspk1avdunyyqul8qyaqqaphkbkokri4ycqrrldqc6eezlctwxccncvhf+dk70rd0f8w/7w6m06jk32hd1jlxhnb+yivwphbmik++ftente4p30t/xd//gv1ffwpy/yh+u//qvm1bm9</latexit> <latexit sha1_base64="y6qd7ctg3gwq+5slaxnajaose5o=">aaadj3icrvllbhmxfpumrxjekszzwesvyojgr4imi0gvbjcykrjpk8wtkcfxjnz4hrlvvi2m8zds+bu2siaqlpktpe2oeh4sc65k6eicc30fdlrozygqb55/5eq16ze2brzu3b5z9157+/6rzusj5ejkojcnebdsq0yoqigwj4wrpi20pi6sl41+fcqnvxn2bhafdfi+y1ssbadhhdveyjelhozrxc3q0i6tad/baww45vgkhweyarndpfnmsrmgexfpluswwct1bvu7grk1m4o78omqsdyoo18vtvnzsvowt6xsyzsskggtj0p8z+3ghouk1tww3sj/v2oe7q7pekiopbgbdo85cwb/v9+jfuwbyuuhreiwbh9k01yuqcxaag7tmjicgoobuellusvkkwsuej6tywcznkobvbfvxomdx0xxnbt3msax7hpgxvnrf2nkne2z9letif+kjuui+0glsqiemyllobjughlcfbo8vuyk0ashuddk9yrfnlsfg/talbeen5piv4ojxpesln39rhpwyrwolfqqpuk7iki9dibeoum0qsj76733pnmf/xf+b/+l/3vp9b1vzgo0ef73h7xcbw0=</latexit> <latexit sha1_base64="y6qd7ctg3gwq+5slaxnajaose5o=">aaadj3icrvllbhmxfpumrxjekszzwesvyojgr4imi0gvbjcykrjpk8wtkcfxjnz4hrlvvi2m8zds+bu2siaqlpktpe2oeh4sc65k6eicc30fdlrozygqb55/5eq16ze2brzu3b5z9157+/6rzusj5ejkojcnebdsq0yoqigwj4wrpi20pi6sl41+fcqnvxn2bhafdfi+y1ssbadhhdveyjelhozrxc3q0i6tad/baww45vgkhweyarndpfnmsrmgexfpluswwct1bvu7grk1m4o78omqsdyoo18vtvnzsvowt6xsyzsskggtj0p8z+3ghouk1tww3sj/v2oe7q7pekiopbgbdo85cwb/v9+jfuwbyuuhreiwbh9k01yuqcxaag7tmjicgoobuellusvkkwsuej6tywcznkobvbfvxomdx0xxnbt3msax7hpgxvnrf2nkne2z9letif+kjuui+0glsqiemyllobjughlcfbo8vuyk0ashuddk9yrfnlsfg/talbeen5piv4ojxpesln39rhpwyrwolfqqpuk7iki9dibeoum0qsj76733pnmf/xf+b/+l/3vp9b1vzgo0ef73h7xcbw0=</latexit> <latexit sha1_base64="y6qd7ctg3gwq+5slaxnajaose5o=">aaadj3icrvllbhmxfpumrxjekszzwesvyojgr4imi0gvbjcykrjpk8wtkcfxjnz4hrlvvi2m8zds+bu2siaqlpktpe2oeh4sc65k6eicc30fdlrozygqb55/5eq16ze2brzu3b5z9157+/6rzusj5ejkojcnebdsq0yoqigwj4wrpi20pi6sl41+fcqnvxn2bhafdfi+y1ssbadhhdveyjelhozrxc3q0i6tad/baww45vgkhweyarndpfnmsrmgexfpluswwct1bvu7grk1m4o78omqsdyoo18vtvnzsvowt6xsyzsskggtj0p8z+3ghouk1tww3sj/v2oe7q7pekiopbgbdo85cwb/v9+jfuwbyuuhreiwbh9k01yuqcxaag7tmjicgoobuellusvkkwsuej6tywcznkobvbfvxomdx0xxnbt3msax7hpgxvnrf2nkne2z9letif+kjuui+0glsqiemyllobjughlcfbo8vuyk0ashuddk9yrfnlsfg/talbeen5piv4ojxpesln39rhpwyrwolfqqpuk7iki9dibeoum0qsj76733pnmf/xf+b/+l/3vp9b1vzgo0ef73h7xcbw0=</latexit> <latexit sha1_base64="y6qd7ctg3gwq+5slaxnajaose5o=">aaadj3icrvllbhmxfpumrxjekszzwesvyojgr4imi0gvbjcykrjpk8wtkcfxjnz4hrlvvi2m8zds+bu2siaqlpktpe2oeh4sc65k6eicc30fdlrozygqb55/5eq16ze2brzu3b5z9157+/6rzusj5ejkojcnebdsq0yoqigwj4wrpi20pi6sl41+fcqnvxn2bhafdfi+y1ssbadhhdveyjelhozrxc3q0i6tad/baww45vgkhweyarndpfnmsrmgexfpluswwct1bvu7grk1m4o78omqsdyoo18vtvnzsvowt6xsyzsskggtj0p8z+3ghouk1tww3sj/v2oe7q7pekiopbgbdo85cwb/v9+jfuwbyuuhreiwbh9k01yuqcxaag7tmjicgoobuellusvkkwsuej6tywcznkobvbfvxomdx0xxnbt3msax7hpgxvnrf2nkne2z9letif+kjuui+0glsqiemyllobjughlcfbo8vuyk0ashuddk9yrfnlsfg/talbeen5piv4ojxpesln39rhpwyrwolfqqpuk7iki9dibeoum0qsj76733pnmf/xf+b/+l/3vp9b1vzgo0ef73h7xcbw0=</latexit> WHY SHOULD RANDOM PROJECTIONS EVEN WORK?! Like = repeating the experiment K times and = averaging y t [k] =x > t u k / p K where each u k [i] = random ± 1 = [ ] [ ] (y s [k] y t [k]) 2 = x > t u k x > s u k 2 /K ky t y s k 2 2 = KX (y s [k] y t [k]) 2 = 1 K KX x > t u k x > s u k 2 k=1 k=1

23 <latexit sha1_base64="3yp9ejgszib0dby1vx1w49yzwpo=">aaadj3icrvllbhmxfpumrxjekszzwesvyojgr4imi0gvbjcykrjpk8wtkcfxjnz4hrlvvi2m8zds+bu2siaqlpktpe2oeh4sc65k6eicc30fdlrozygqb55/5eq16ze2brzu3b5z9157+/6rzusj5ejkojcnebdsq0yoqigwj4wrpi20pi6sl41+fcqnvxn2bhafdfi+y1ssbadhhdveygexprzmuvwt6tcokwa/wwsmoobxpichmgkzw6x5zektbnlxas7rmfnldqb7u4eznzudu/dpklewo1+vtfnysvowtyxsyzsskggtj0p8z93ghouk1tww3sj/v1oe7q7pekiopbgbdo85cwb/v9+jfuwbyuuhreiwbh9k01yuqcxaag7tmjicgoobuellusvkkwsuej6tywcznkobvbfvxomdx0xxnbt3msax7hpgxvnrf2nkne2z9letif+kjuui+0glsqiemyllobjughlcfbo8vuyk0ashuddk9yrfnlsfg/talbeen5piv4ojxpesln39rhpwyrwolfqqpuk7iki9dibeoum0qsj76733pnmf/xf+b/+l/3vp9b1vzgo0ef73h6bfbw0=</latexit> <latexit sha1_base64="3yp9ejgszib0dby1vx1w49yzwpo=">aaadj3icrvllbhmxfpumrxjekszzwesvyojgr4imi0gvbjcykrjpk8wtkcfxjnz4hrlvvi2m8zds+bu2siaqlpktpe2oeh4sc65k6eicc30fdlrozygqb55/5eq16ze2brzu3b5z9157+/6rzusj5ejkojcnebdsq0yoqigwj4wrpi20pi6sl41+fcqnvxn2bhafdfi+y1ssbadhhdveygexprzmuvwt6tcokwa/wwsmoobxpichmgkzw6x5zektbnlxas7rmfnldqb7u4eznzudu/dpklewo1+vtfnysvowtyxsyzsskggtj0p8z93ghouk1tww3sj/v1oe7q7pekiopbgbdo85cwb/v9+jfuwbyuuhreiwbh9k01yuqcxaag7tmjicgoobuellusvkkwsuej6tywcznkobvbfvxomdx0xxnbt3msax7hpgxvnrf2nkne2z9letif+kjuui+0glsqiemyllobjughlcfbo8vuyk0ashuddk9yrfnlsfg/talbeen5piv4ojxpesln39rhpwyrwolfqqpuk7iki9dibeoum0qsj76733pnmf/xf+b/+l/3vp9b1vzgo0ef73h6bfbw0=</latexit> <latexit sha1_base64="3yp9ejgszib0dby1vx1w49yzwpo=">aaadj3icrvllbhmxfpumrxjekszzwesvyojgr4imi0gvbjcykrjpk8wtkcfxjnz4hrlvvi2m8zds+bu2siaqlpktpe2oeh4sc65k6eicc30fdlrozygqb55/5eq16ze2brzu3b5z9157+/6rzusj5ejkojcnebdsq0yoqigwj4wrpi20pi6sl41+fcqnvxn2bhafdfi+y1ssbadhhdveygexprzmuvwt6tcokwa/wwsmoobxpichmgkzw6x5zektbnlxas7rmfnldqb7u4eznzudu/dpklewo1+vtfnysvowtyxsyzsskggtj0p8z93ghouk1tww3sj/v1oe7q7pekiopbgbdo85cwb/v9+jfuwbyuuhreiwbh9k01yuqcxaag7tmjicgoobuellusvkkwsuej6tywcznkobvbfvxomdx0xxnbt3msax7hpgxvnrf2nkne2z9letif+kjuui+0glsqiemyllobjughlcfbo8vuyk0ashuddk9yrfnlsfg/talbeen5piv4ojxpesln39rhpwyrwolfqqpuk7iki9dibeoum0qsj76733pnmf/xf+b/+l/3vp9b1vzgo0ef73h6bfbw0=</latexit> <latexit sha1_base64="3yp9ejgszib0dby1vx1w49yzwpo=">aaadj3icrvllbhmxfpumrxjekszzwesvyojgr4imi0gvbjcykrjpk8wtkcfxjnz4hrlvvi2m8zds+bu2siaqlpktpe2oeh4sc65k6eicc30fdlrozygqb55/5eq16ze2brzu3b5z9157+/6rzusj5ejkojcnebdsq0yoqigwj4wrpi20pi6sl41+fcqnvxn2bhafdfi+y1ssbadhhdveygexprzmuvwt6tcokwa/wwsmoobxpichmgkzw6x5zektbnlxas7rmfnldqb7u4eznzudu/dpklewo1+vtfnysvowtyxsyzsskggtj0p8z93ghouk1tww3sj/v1oe7q7pekiopbgbdo85cwb/v9+jfuwbyuuhreiwbh9k01yuqcxaag7tmjicgoobuellusvkkwsuej6tywcznkobvbfvxomdx0xxnbt3msax7hpgxvnrf2nkne2z9letif+kjuui+0glsqiemyllobjughlcfbo8vuyk0ashuddk9yrfnlsfg/talbeen5piv4ojxpesln39rhpwyrwolfqqpuk7iki9dibeoum0qsj76733pnmf/xf+b/+l/3vp9b1vzgo0ef73h6bfbw0=</latexit> <latexit sha1_base64="piwkdkn7kx4tj0ajldd+otvtvhq=">aaacy3icdvfba9rafj5erxwrnq3erhi4cj7wtcl2exckxgqvfdy2kmqwmx3pdptj0pkx7rqyp9kbny/+dyfbrvtrbwmf3/e9n2++yepcwqrd755/6/adjbub9wzb9x883a52dk9s1rije1kvltnlhmvcltghrqwe1qafzgo8zezve/30mxqrqvijlwpmtdgvva6kieelwddyc5plebvouormcbyga+bsmz9iqurftnolc3gz2wtd7ftuuvxcthhjrrdfzmgqumgzddfdkvpnxlvaihja6q7iwgnpg2e4csoqcw494aevqgcod8d7fay8l1wn2bqo0+bbpk1ko7ekwqhrix7wlltckjifdoo4svglorfngdlyco02avczdfdcmvpik+nosbbib3a0qlu70jlz9vvbv7we/jcwnzspk1avdunyyqul8qyaqqaphkbkokri4ycqrrldqc6eezlctwxccncvhf+dk70rd0f8w/7w6m06jk32hd1jlxhnb+yivwphbmik++ftente4p30t/xd//gv1ffwpy/yh+u//qvm1bm9</latexit> <latexit sha1_base64="piwkdkn7kx4tj0ajldd+otvtvhq=">aaacy3icdvfba9rafj5erxwrnq3erhi4cj7wtcl2exckxgqvfdy2kmqwmx3pdptj0pkx7rqyp9kbny/+dyfbrvtrbwmf3/e9n2++yepcwqrd755/6/adjbub9wzb9x883a52dk9s1rije1kvltnlhmvcltghrqwe1qafzgo8zezve/30mxqrqvijlwpmtdgvva6kieelwddyc5plebvouormcbyga+bsmz9iqurftnolc3gz2wtd7ftuuvxcthhjrrdfzmgqumgzddfdkvpnxlvaihja6q7iwgnpg2e4csoqcw494aevqgcod8d7fay8l1wn2bqo0+bbpk1ko7ekwqhrix7wlltckjifdoo4svglorfngdlyco02avczdfdcmvpik+nosbbib3a0qlu70jlz9vvbv7we/jcwnzspk1avdunyyqul8qyaqqaphkbkokri4ycqrrldqc6eezlctwxccncvhf+dk70rd0f8w/7w6m06jk32hd1jlxhnb+yivwphbmik++ftente4p30t/xd//gv1ffwpy/yh+u//qvm1bm9</latexit> <latexit sha1_base64="piwkdkn7kx4tj0ajldd+otvtvhq=">aaacy3icdvfba9rafj5erxwrnq3erhi4cj7wtcl2exckxgqvfdy2kmqwmx3pdptj0pkx7rqyp9kbny/+dyfbrvtrbwmf3/e9n2++yepcwqrd755/6/adjbub9wzb9x883a52dk9s1rije1kvltnlhmvcltghrqwe1qafzgo8zezve/30mxqrqvijlwpmtdgvva6kieelwddyc5plebvouormcbyga+bsmz9iqurftnolc3gz2wtd7ftuuvxcthhjrrdfzmgqumgzddfdkvpnxlvaihja6q7iwgnpg2e4csoqcw494aevqgcod8d7fay8l1wn2bqo0+bbpk1ko7ekwqhrix7wlltckjifdoo4svglorfngdlyco02avczdfdcmvpik+nosbbib3a0qlu70jlz9vvbv7we/jcwnzspk1avdunyyqul8qyaqqaphkbkokri4ycqrrldqc6eezlctwxccncvhf+dk70rd0f8w/7w6m06jk32hd1jlxhnb+yivwphbmik++ftente4p30t/xd//gv1ffwpy/yh+u//qvm1bm9</latexit> <latexit sha1_base64="piwkdkn7kx4tj0ajldd+otvtvhq=">aaacy3icdvfba9rafj5erxwrnq3erhi4cj7wtcl2exckxgqvfdy2kmqwmx3pdptj0pkx7rqyp9kbny/+dyfbrvtrbwmf3/e9n2++yepcwqrd755/6/adjbub9wzb9x883a52dk9s1rije1kvltnlhmvcltghrqwe1qafzgo8zezve/30mxqrqvijlwpmtdgvva6kieelwddyc5plebvouormcbyga+bsmz9iqurftnolc3gz2wtd7ftuuvxcthhjrrdfzmgqumgzddfdkvpnxlvaihja6q7iwgnpg2e4csoqcw494aevqgcod8d7fay8l1wn2bqo0+bbpk1ko7ekwqhrix7wlltckjifdoo4svglorfngdlyco02avczdfdcmvpik+nosbbib3a0qlu70jlz9vvbv7we/jcwnzspk1avdunyyqul8qyaqqaphkbkokri4ycqrrldqc6eezlctwxccncvhf+dk70rd0f8w/7w6m06jk32hd1jlxhnb+yivwphbmik++ftente4p30t/xd//gv1ffwpy/yh+u//qvm1bm9</latexit> <latexit sha1_base64="y6qd7ctg3gwq+5slaxnajaose5o=">aaadj3icrvllbhmxfpumrxjekszzwesvyojgr4imi0gvbjcykrjpk8wtkcfxjnz4hrlvvi2m8zds+bu2siaqlpktpe2oeh4sc65k6eicc30fdlrozygqb55/5eq16ze2brzu3b5z9157+/6rzusj5ejkojcnebdsq0yoqigwj4wrpi20pi6sl41+fcqnvxn2bhafdfi+y1ssbadhhdveyjelhozrxc3q0i6tad/baww45vgkhweyarndpfnmsrmgexfpluswwct1bvu7grk1m4o78omqsdyoo18vtvnzsvowt6xsyzsskggtj0p8z+3ghouk1tww3sj/v2oe7q7pekiopbgbdo85cwb/v9+jfuwbyuuhreiwbh9k01yuqcxaag7tmjicgoobuellusvkkwsuej6tywcznkobvbfvxomdx0xxnbt3msax7hpgxvnrf2nkne2z9letif+kjuui+0glsqiemyllobjughlcfbo8vuyk0ashuddk9yrfnlsfg/talbeen5piv4ojxpesln39rhpwyrwolfqqpuk7iki9dibeoum0qsj76733pnmf/xf+b/+l/3vp9b1vzgo0ef73h7xcbw0=</latexit> <latexit sha1_base64="y6qd7ctg3gwq+5slaxnajaose5o=">aaadj3icrvllbhmxfpumrxjekszzwesvyojgr4imi0gvbjcykrjpk8wtkcfxjnz4hrlvvi2m8zds+bu2siaqlpktpe2oeh4sc65k6eicc30fdlrozygqb55/5eq16ze2brzu3b5z9157+/6rzusj5ejkojcnebdsq0yoqigwj4wrpi20pi6sl41+fcqnvxn2bhafdfi+y1ssbadhhdveyjelhozrxc3q0i6tad/baww45vgkhweyarndpfnmsrmgexfpluswwct1bvu7grk1m4o78omqsdyoo18vtvnzsvowt6xsyzsskggtj0p8z+3ghouk1tww3sj/v2oe7q7pekiopbgbdo85cwb/v9+jfuwbyuuhreiwbh9k01yuqcxaag7tmjicgoobuellusvkkwsuej6tywcznkobvbfvxomdx0xxnbt3msax7hpgxvnrf2nkne2z9letif+kjuui+0glsqiemyllobjughlcfbo8vuyk0ashuddk9yrfnlsfg/talbeen5piv4ojxpesln39rhpwyrwolfqqpuk7iki9dibeoum0qsj76733pnmf/xf+b/+l/3vp9b1vzgo0ef73h7xcbw0=</latexit> <latexit sha1_base64="y6qd7ctg3gwq+5slaxnajaose5o=">aaadj3icrvllbhmxfpumrxjekszzwesvyojgr4imi0gvbjcykrjpk8wtkcfxjnz4hrlvvi2m8zds+bu2siaqlpktpe2oeh4sc65k6eicc30fdlrozygqb55/5eq16ze2brzu3b5z9157+/6rzusj5ejkojcnebdsq0yoqigwj4wrpi20pi6sl41+fcqnvxn2bhafdfi+y1ssbadhhdveyjelhozrxc3q0i6tad/baww45vgkhweyarndpfnmsrmgexfpluswwct1bvu7grk1m4o78omqsdyoo18vtvnzsvowt6xsyzsskggtj0p8z+3ghouk1tww3sj/v2oe7q7pekiopbgbdo85cwb/v9+jfuwbyuuhreiwbh9k01yuqcxaag7tmjicgoobuellusvkkwsuej6tywcznkobvbfvxomdx0xxnbt3msax7hpgxvnrf2nkne2z9letif+kjuui+0glsqiemyllobjughlcfbo8vuyk0ashuddk9yrfnlsfg/talbeen5piv4ojxpesln39rhpwyrwolfqqpuk7iki9dibeoum0qsj76733pnmf/xf+b/+l/3vp9b1vzgo0ef73h7xcbw0=</latexit> <latexit sha1_base64="y6qd7ctg3gwq+5slaxnajaose5o=">aaadj3icrvllbhmxfpumrxjekszzwesvyojgr4imi0gvbjcykrjpk8wtkcfxjnz4hrlvvi2m8zds+bu2siaqlpktpe2oeh4sc65k6eicc30fdlrozygqb55/5eq16ze2brzu3b5z9157+/6rzusj5ejkojcnebdsq0yoqigwj4wrpi20pi6sl41+fcqnvxn2bhafdfi+y1ssbadhhdveyjelhozrxc3q0i6tad/baww45vgkhweyarndpfnmsrmgexfpluswwct1bvu7grk1m4o78omqsdyoo18vtvnzsvowt6xsyzsskggtj0p8z+3ghouk1tww3sj/v2oe7q7pekiopbgbdo85cwb/v9+jfuwbyuuhreiwbh9k01yuqcxaag7tmjicgoobuellusvkkwsuej6tywcznkobvbfvxomdx0xxnbt3msax7hpgxvnrf2nkne2z9letif+kjuui+0glsqiemyllobjughlcfbo8vuyk0ashuddk9yrfnlsfg/talbeen5piv4ojxpesln39rhpwyrwolfqqpuk7iki9dibeoum0qsj76733pnmf/xf+b/+l/3vp9b1vzgo0ef73h7xcbw0=</latexit> WHY SHOULD RANDOM PROJECTIONS EVEN WORK?! Like = repeating the experiment K times and = averaging y t [k] =x > t u k / p K where each u k [i] = random ± 1 = [ ] [ ] (y s [k] y t [k]) 2 = x > t u k x > s u k 2 /K ky t y s k 2 2 = KX (y s [k] y t [k]) 2 = 1 K KX x > t u k x > s u k 2 k=1 k=1 This is an average over K trials

24 WHY SHOULD RANDOM PROJECTIONS EVEN WORK?! For large K, not only true in expectation but also with high probability For any > 0, if K log (n ) 2, with probability 1 W, for all pairs of data points i, j {1,...,n}, over draw of 2 2 (1 ) y i y j 2 x i x j 2 (1 + ) y i y j 2

25 WHY SHOULD RANDOM PROJECTIONS EVEN WORK?! For large K, not only true in expectation but also with high probability For any > 0, if K log (n ) 2, with probability 1 W, for all pairs of data points i, j {1,...,n}, over draw of 2 2 (1 ) y i y j 2 x i x j 2 (1 + ) y i y j 2 Lets try on Matlab...

26 WHY SHOULD RANDOM PROJECTIONS EVEN WORK?! For large K, not only true in expectation but also with high probability For any > 0, if K log (n ) 2, with probability 1 W, for all pairs of data points i, j {1,...,n}, over draw of 2 2 (1 ) y i y j 2 x i x j 2 (1 + ) y i y j 2 Lets try on Matlab... This is called the Johnson-Lindenstrauss lemma or JL lemma for short.

27 WHY IS THIS SO RIDICULOUSLY MAGICAL? n= 1000 d = 1000

28 WHY IS THIS SO RIDICULOUSLY MAGICAL? n= 1000 d = 1000 If we take K = 69.1/ 2, with probability 0.99 distances are preserved to accuracy

29 WHY IS THIS SO RIDICULOUSLY MAGICAL? n= 1000 d = If we take K = 69.1/ 2, with probability 0.99 distances are preserved to accuracy

30 WHY IS THIS SO RIDICULOUSLY MAGICAL? n= 1000 d = If we take K = 69.1/ 2, with probability 0.99 distances are preserved to accuracy

31 TWO VIEW DIMENSIONALITY REDUCTION Data comes in pairs (x 1, x 1 ),...,(x n, x n) where x t s are d dimensional and x t s are d dimensional Goal: Compress say view one into y 1,...,y n, that are K dimensional vectors Retain information redundant between the two views Eliminate noise specific to only one of the views

32 Canonical Correlation Analysis z y x

33 Canonical Correlation Analysis Age + Gender z y Candies per week x

34 EXAMPLE I: SPEECH RECOGNITION + Audio might have background sounds uncorrelated with video Video might have lighting changes uncorrelated with audio Redundant information between two views: the speech

35 EXAMPLE II: COMBINING FEATURE EXTRACTIONS Method A and Method B are both equally good feature extraction techniques Concatenating the two features blindly yields large dimensional feature vector with redundancy Applying techniques like CCA extracts the key information between the two methods Removes extra unwanted information

36 How do we get the right direction? (say K = 1)

37 How do we get the right direction? (say K = 1) Age + Gender Candies per week

38 WHICH DIRECTION TO PICK? View I View II

39 WHICH DIRECTION TO PICK? View I View II

40 WHICH DIRECTION TO PICK? View I View II

41 WHICH DIRECTION TO PICK? View I View II

42 WHICH DIRECTION TO PICK? PCA direction 0 0

43 WHICH DIRECTION TO PICK? Direction has large covariance

44 How do we pick the right direction to project to?