TSBB15 Computer Vision Lecture 10 Recap. of TSBB06, Maximum Likelihood and RANSAC!1
Prelude to Project 2 Image credit: Bundler home page http://www.cs.cornell.edu/~snavely/bundler/ Project 2: Start with two views, and successively add more. Finally perform simultaneous refinement of 3D point positions and camera positions (Bundle adjustment).!2
Prelude to Project 2 Highlights of differences compared to Project 1: Less programming More math skills More ways to get lost in 3D Unit testing (also of external functions) even more important!3
<latexit sha1_base64="y5nnmdljvphyyn3oxc3scmkxfks=">aaac9hicjvhlssnafd2n73fvpztgevzvpaq6euq3lhvsfyyujj3woxkxmrrd6we4cydu/qg3+g3ih+hfegem4gprcunonhvpmbn3eknau2lztyvjahhkdgx8ynjqemz2rjy/0ejjtpis7sdble48n2ubj1hdchmwk0qwn/qcdux191t8umdeyuposoyjowvdtstb3hcluc3ywt/x2my+2hy81ufrpwldkfjfig/ajpm3a45joyxqffdncswqwnqzp4fdgaqkdrcxh+gghrg+morgicajb3cr0nmkgxys4s7qj04q4jrommakatpkypthetulb4d2pwub0v55plrt0ykbvykujlzie1oeikxom3u8086k/c27rz3v3xl6e4vxskzeobf/6d4z/6tttui0savr4frtohlvnv+4zlor6ubmp6okostekdyiucdsa+v7n02tsxxtqreujj/rtmwqvv/kznhrt6qb29/h+rm0alv7vwofblr2dotrj2mjy1ilew5ib/s4qj28l3ghezwypepkudzu3lknuqfzxjdl3l4cfhojmq==</latexit> Recap: 2D homogeneous coordinates A 2D point (y 1, y 2 ) is given a homogeneous representation as: y 2 R 3 <latexit sha1_base64="6rpkq7ulxdjvawz4wmqflnkvlhi=">aaac23icjvhlssnafd2nr1pfvcgnm2arxjxecrosunfzxt6grsvjp+3qvegmqqlduro3/obb/r/xd/qvvdnguivohcrnzr3nznx77ddlstcml4w2mzs3v5bdzc0tr6yu5dc3angqra6rooebra3bipnlfvyvxliseubm8myx1e3hiyzxr1gu88c/ekoqtt2r7/medyxbvce/nw7zpx00axg/5vliynvj88llqzmvgevdlx0amckoif2vip+mfroi4ccbbwyfgralcze9tzgweblxxpi4ibbxcyyjcqrnkitrhkxskl592jvt1qe99iyv2qftxhojuuryju1aerfhezqu4olyluxv3mplke82or+dennecgyi/uv3mflfnaxfoicjvqonmklfyoqc1cvrxze3179ujcghje7ilsujwo5sfvzzv5py1s57a6n4q8qurnw7aw6cn3llgrd5c5ztolzfnetf8+ygud5or53fnnawr/m8rbmnqkbk3td4wcoetlz2o91qdx+pwibvbolb0u7faaevmjg=</latexit> 0 1 y = @ y 1 y 2 1 A Any scalar multiple of this v is also a homogeneous representation of (y 1, y 2 ) Canonical form of homogeneous coordinates for a 2D point We will use y to refer to the 2D point as well as its homogeneous coordinates!!4
<latexit sha1_base64="rpncfkd/8zbvmcob7qeqiqohxzy=">aaadbnicjvhnbtqwgjwg+gstcxy5rkyqulbkkbjckhw4ccwsu61uv5xj9e5adelidqpwq73zjty4ia68afdwvaf9cz6blijwfthkmp5vzuzplmqtnm+y70vjjzvlk6tr6xu3bm9u3encvtdwprfc9oxrxh4u3emtktn3ymt5ufvjy0ll/elkdajvn0rrlkne+fnahpv8xkmretwtddx5owxfknwzhvbisaqmdcm9vwczjoxjxnctzhhzqlrgj8xodjpv8lfyunpnelkc6vwqt6clduyzzhwmqxgincghucet1ubw9bwir4aaucnmiboevkxlzlbb3ozukhsc2bp6jml22livzuomi25bq2h6ltltpckpiz0lhfzly72jyyg9lnsam8pezulftfklsr4tyv/lwyj/1xd68rjhrexbuu91zej3ok1p4qmenad/douposyu4chvlwernytztqphxd7d2fjy/xgvgq1z0wobxird0gxnl6/zkhg87exbvfzts+7uq/aq1/aad/gy7vm5dvege+ht9gd8xtfmk/fjx+rt8vmxnflqpffx10i+/asw7qvz</latexit> Recap: 2D homogeneous coordinates A 2D line, at distance d from the origin, and with (cos α, sin α) as a normal vector is given a (dual) homogeneous representation as l 2 R 3 : <latexit sha1_base64="fcvyrxd8pjw4nhrmj5klpppzfmm=">aaac23icjvhlssnafd2nr1pfvcgnm2arxjxecrosunfzxt6grsvjp+3qvegmqqlduro3/obb/r/xd/qvvdnguivohcrnzr3nznx77ddlstcml4w2mzs3v5bdzc0tr6yu5dc3angqra6rooebra3bipnlfvyvxliseubm8myx1e3hiyzxr1gu88c/ekoqtt2r7/medyxbvce/nw7zpd2dtljf8iwxso3x+esy1mkxjkkhlj4nzbqukk5kkh9gc10ecjdaa4mpqdifhziejkwycilry0xcriirommeodimlmuowyj2sn8+7zop69neeszk7daplr0rkxxskiagviiwpe1x8uq5s/y377hylhcb0d9ovtxibqbe/qx7zpyvttyi0morqofttafizhvo6pkorsib61+qeuqqeidxl+iryucpp/usk02sape9tvt8vwvkvu6dndfbm7wlddj8oc5punsvmqwiexzqkb+no85igzvyo3keooxtvfal72s84bfpwlu70w61u49ulznqnvftaffvh/+yiw==</latexit> 0 1 cos l = @ sin A Canonical form of homogeneous coordinates for a 2D line Any scalar multiple of this l is also a homogeneous representation of the same line We will use l to refer to the 2D line as well as its homogeneous coordinates!!5
<latexit sha1_base64="pgiw1xe1oworvmfakfnfetjjsm0=">aaac2nicjvhlssnafd2nr1pfvxhljlgevyvrqzdfny4r2ac0psttaq3ni2qillybd+lwh3crhyt+gf6fd8yu1ci6icmzc+85m/dej/lcrbjga06bm19yxmovf1zw19y3iptb9srmy8zrlptcuonycffcgneekzzejgju+47hg87gxmybnzxo3dc4esoit327h7g9l9mcqe5xzzyynj4+nfisgwqfvcm4uywzzumtfraygsghw9ww+ailxyrgsogdi4ag7mfgqk8ljgxexluxii4m5ko4xwqf0qauxsndjnza3z7twhkb0f56jkrn6bsp3piuovzje1jetfiepqt4qpwl+5v3shnkuw3p72reprec18t+pztm/lcnaxho4vtv4fjnkwjkdsxzsvvx5m31l1ujcoiik7hl8zgwu8ppn3wlsvttsre2ir+ptmnkpctyu7zlw9kazz/jnax1w7j5vdyvj0uvs2zueexidwc0zxnucieqauq9wioe8kxz2q12p91/pmq5tlonb0t7+aad3phc</latexit> Recap: 2D homogeneous coordinates From these homogeneous representations follow: Point y lies on line l, y l =0 <latexit sha1_base64="ey3nrz2hqyzxolvth/ssy+zez5y=">aaac63icjvhltttafd0xtirhs4alg4uoifvkt5vaqkiibliwskwgigge7mkkgcx2womxkiryb+zyibb9abbwh4g/gl/onakjtusohcv2mxpvotp33iilra4976nizmy+ez9xnv9yxprwcbm2snqcy0ix3miyluokcnmei5s3tnaxp8kud5mo5u1o8m3e2xdc5ukmp/qw42dj2etfv7bqe3ve2wyoefcr0evrucl5geyogqjrdscb60htctze885rda/h2evoa78edzsrkwupcncbbeobbbwpnoeyixj6tuhdq0bcgubekulcxjngwcbtqvmcmkjib/tt0e60zfpag8/cqhmdetorsonie2kk5snc5jtxxgvrbni3vefw09xtsp+o9eqi1egt+y/djpn/daywjs52ba2casosy6pjputhu2ju7v5rlsahjdidoxrxhjlvtvrswk1uaze9dw382wya1uxzmvvgxdysbuy/huc0op7c8l80/o9f6/sh5airwmcgtmie29jhizpokfcv7ncpbydxrp0b5/z3qlmpnwv4azk/fwedej9c</latexit> Point y intersects lines l 1 and l 2, y = l 1 l <latexit sha1_base64="vagyh5ccnfq8+9/hbkg2thsj3ms=">aaac9nicjvhlssnafd2nr1pfvzdugkvwvrivfeqounhhqsgqyepj0mkdzivjrc2l3+honbj1b9zqj4h/oh/hntefh4hosobcc+85mtvxswkesst6krhdwyojy8xx0stk1prmexbuki0z4bo6hwexophclau8ynxjzcboeshc0avysxe+o/lhf0ykpi4oztdhjddtrlznfvcs1szbzh5rs8e7z9ivir50nnuo1za7/s29b/2m7ugesnqqrjtlfatq6wx+bhyoksjxflx+homwyvjieiihgiqcwevkzylsweiia6bhncdedz6hjxjpm6pivoese07fdkwnortrrdxtrfbplwg9gpqmlkgtu50grp5m6nymnrx7m3dpe6qzdwn3cq+qwikzyv/sdsr/q1o9slsxoxvg1foigdwdn7tk+lbuyc1pxulysihtuev5qdjxyse9m1qt6t7v3bo6/6orfatip6/n8kzosqo2v4/zjzhaqdqrvftgrvlbzkddxaiwsuzzxecnu9hhnbyv8ybhpblxxo1xa9x9lbqfxdopl8u4fwcddkpb</latexit> 2 Line l intersects points y 1 and y 2, l = y 1 y <latexit sha1_base64="bwmvozdoyrtuw2na9cqq7lawuvs=">aaac9nicjvhlssnafd2nr1pfvzdugkvwvrivfeqounhhqsgqyepj0mkdzivjrc2l3+honbj1b9zqj4h/oh/hntefh4hosobcc+85mtvxswkesst6krhdwyojy8xx0stk1prmexbuki0z4bo6hwexophclau8ynxjzcboeshc0avysxe+o/lhf0ykpi4oztdhjddtrlznfvcs1szbzh5rs8e7z9ivir50nnuo1zad/pbeu/2m7ugesnqqrjtlfatq6wx+bhyoksjxflx+homwyvjieiihgiqcwevkzylsweiia6bhncdedz6hjxjpm6pivoese07fdkwnortrrdxtrfbplwg9gpqmlkgtu50grp5m6nymnrx7m3dpe6qzdwn3cq+qwikzyv/sdsr/q1o9slsxoxvg1foigdwdn7tk+lbuyc1pxulysihtuev5qdjxyse9m1qt6t7v3bo6/6orfatip6/n8kzosqo2v4/zjzhaqdqrvftgrvlbzkddxaiwsuzzxecnu9hhnbyv8ybhpblxxo1xa9x9lbqfxdopl8u4fwc8e6po</latexit> 2 The distance from point y to line l is given by y l if they both are in canonical form!6
Recap: 2D homogeneous coordinates Rigid transformations: translation + rotation Scaling Affine transformations Projective transformations (homographies) Can be represented as linear mappings of the homogeneous coordinates: y 0 = Hy <latexit sha1_base64="uxbrh9oojqvug2daqyzctlxv6hi=">aaac3hicjvhlssnafd2nr1pfurcu3asl6kokkuhgklrpsoj9qftlkk5rajqezcku0p07cespunxvef9a/8i74xtuijohybnn3npnppzi9xjumq8zbwz2bn4hu5hbwl5zxdpxn6pjmmyuq7ihh8z1x06y7wwswj3us3oum3vg+kzm9m+fv3bd4sqlg0s+jfhrypccr+u5nieqrw+nmk7xgi73tiuojzxd1vnmwzthmaawanmouw71fztrqqgxkqzgcmaj+7cr0noabrmrcs2miisjedlpmeaotclfmyqwie3tt0dwq7eb2sjnituu3eltg5pswc5pqoqlcyvbdolpzwbb/pz7jhok2ob0d1suabec18t+pzte/lcneuho4kt24ffpkwred67kksqpimqnl11xyharj3ch/dfhvyonczakjpg9i9na0v8miwurbfffpngxvdkcrz/rnabvg4j1wlaujvlfm7xqllaxg33a5zgkkkgmiqz/eu941q60w+1ou/8m1tjks4lvr3v4aegnmgc=</latexit> l 0 = H T l <latexit sha1_base64="ndl+qztj2ca1zunylcc0kwsgbgq=">aaac4xicjvhlssnafd3gv62vqetdbivoxpkoobuh6kzlbwslviwztju0tuiyeuroxp07cespunwfef9a/8i74xr8idohyzlz7zkz914vdvxu2pblkde8mjo2xpgotk5nz8yac/phazqljndyferjw3nthvghrwlfblwrj9zteggve519ga9f8st1o/bi9gj+2nuvqr/tm1cqdw4u5u2vbqx91v0fqv2zfp2or8lzs2sxbbwsn8droas9dilzgu20eiehqxcciqthac5sek7gwezm3cly4hjcvopz9fekbuzzndjcyjv0vaddiwzd2kvpvkkznrlqm5dswgppisplcmvtlbxpllnkf/polae8w4/+nvbqeitwsexfukhmf3wyfoe2dlqnptuuk0zwx7rlproib259qkqqq0ycxc2kj4szug76bclnqmqxvxvv/fvlslbumc7n8czvsqn2vo/zjzjekdubzedwq1tz06muybhlwkn5bqockg5qi+9rpoartwyzboxb4+4j1rjsmgv8wcb9o0xhmly=</latexit> Dual transformations of lines!!7
<latexit sha1_base64="3/v7tg+a9kmorma7u/h5g3oxg4m=">aaac8xicjvhlssnafd3g9zvq0k2wcc6kjcrorii6calgq9cwkqrthcylzeqopt/hzp249qfc6k+if6b/4z1xcmornzdk3hpvotn3bpbfxejxfr2xrsfgjyanpmdm5+yxfu2l5zpiizxk1tcn0vw88awlemkqksuinwc58+mgymfb1ahkn12zxpa0ozxdjdvj/ylhhr76kqiwvdlrbb3h7zckj5nqqddfrxu22frgt+ysw3b1coabz0ajzh2n9gsaacnfiaixgbjiwhf8chrq8oaii66jhne5ia7zdh3mklagkkyvprfx9l2gqg7yhgllkbq6pf0iennsolgntup1owg1m6pzhxzw7g/epe2pztalf2c8ymillon9szeo/k9o9slrwz7ugvnpmwzud6fxkfstqjm7x7qs5jarp3cb8jnhucsh9+xojdc9q7v1df5nvypwxagplfcutkkd9n6ocxjutsredtk72slvdsyop7cknwzqphdrwrgousxvgzzicc+wsg6to+v+s9qamzovffvwwwefm6gt</latexit> <latexit sha1_base64="tdolqxp8ollyotl/rtdvmx2yuxy=">aaac1xicjvhlssnafd2nr1pfuzdugkvwvrivdfl047kcbyw2lcsd1qf5kzkusulo3podbvwxxd/qv/domijarcckoxpupwfm3uslarfstl8lxsli0vjkcbw0tr6xuwvu7zrenku+q/txekc3nitywcnwl1wg7czjmrt6awt6wwsvb45ykngcxctxwjqho4h4n/uujkprmq1j2+tb7rttbusemte1y3bf1suab04oyshxltzf0eypmxxkcmeqqrio4elq04idgwlxhuyiswlxhweyoktajliyzbjeduk7of0rzypak0+h1t6detcbktlcawliyksjq9mshc+0s2j/855ot3w3mf293cskvukw2l90s8z/6lqten2c6ro41zrorlxn5y6z7oq6ufwlkkkocxek9yieeva1ctzns2uerl311txxn52pwlx389wm7+qwngdn5zjnqeoo4hxxnkutcvu8h3ure9jhic3zffvcooy6ey/wicc8g01jatwz95+prihx7olbmh4+ap1zlge=</latexit> Recap: Cross product operator The cross product between vectors a and b: a b <latexit sha1_base64="sz1wwrz1lvv1c+ugebz7rc4ike0=">aaac2xicjvhlssnafd2nr1pf9bfzeyycq5koomuig5cvbcu0pstpta7ni8xeqkuld+lwh3crpyt+gf6fd8yu1ci6icm5595zzu5cn/a5kjb1mjnmzufmf/klhaxlldw14vpgxurp4rgaf/lrcuk6gvk8zdxjpc8u44q5geuzhjs4vfngnusej8iloyxzo3d6ie9xz5fedypbo5bbm51xs/kacr24406xzjutvcxpygeghgxvo+ilwugigocuarhcsmi+hah6mrbhisaujrfxcsgu8wxjfeibuhwjcofyax37fduznqryeqqt9mgxn96elcz2srnrxujy7wbqfkqdffub90h7qrmn6e9mxggxelfe/qwbvp5xp3qr6ofy98cpp1gzqjsvc0n1raitm1+6kuqqe6dwl/ijyu8rj/dsao3qvau7dxt+tvcqvsvevpvixz2sbmz/hoc0qo+x7yoyfx5yqpxko85jgzvyo3keoyizvfej7xs84gnprto4ne6m+89si5dpnvftgq8futgxsa==</latexit> can sometimes be written more conveniently as a 3 3 matrix [a] applied to b: a b =[a] b!8
Recap: 3D homogeneous coordinates Homogeneous coordinates for 2D points and lines can be extended in a straightforward way to homogeneous coordinates for 3D points x and planes p Also 3D lines can be given a homogeneous representation Parameter form: t x 1 + (t 1) x 2 Plücker coordinates: L = x 1 x 2 T x 2 x 1 T!9
<latexit sha1_base64="vwmnquala/yhko44z0rpyc7ddke=">aaac3nicjvhlssnafd2nr1pfvvfijlgevyvrqzffblxwsa9os03sar2af8lelkw4cydu/qg3+jnih+hfegdmqs2iezjz5tx7zsyda4cuj4vhvga0mdm5+yxsym5pewv1lb++uyudjhjy1qncigryvsxc7roq4mjljtbilme7rg4pyjjev2jrzap/xaxd1vasvs973leeuz381qhl9/thubvzt8hywc3x406+ybqnnfrpykagghrugvwlwugigimehhh8cmiulmt0nwhcqehcgypiikjcxrngyje2osxggraxa5r7tgumre976rkrtuonuprhpnsxs5qa8ilc8jrdxrpllnnfvefku95tskudenneclws+5dukvlfnaxfoidjvqonmklfyoqc1cvrryjvrn+pspbdsjzexyphhb2lnlyzrjsxql2+raxibyptsnlvplkj3uutqchmz3zog9p+0twommehhdjj2uostrgdperneuo4rqvv8r7bi57wrf1ot9qddv+zqmvszsa+de3ha5l9mby=</latexit> Recap: Pinhole camera The pinhole camera maps 3D points to a 2D image: y Cx C is the 3 4 camera (projection) matrix Does not represent geometric distortion from the camera lens!10
<latexit sha1_base64="uiniftbihjscxz8yeavh089uh1o=">aaad8nicjvflbtraec3hfmlwyqswbcxgibzo5laryimukq3lidfjpdiy2j3torx/1g4jimtowy4dyssf2mihedeaw1bdauoyez+22n716r1yvxfwfko1yfjv2/avxlx0efpk6oq16ze2xts399u600lorf3u+jdjrsxujwdgmuienlrymivkqxaya/mhr6ruvv29mkenpc75olk5etwglw57d/oky4pd5zmkkwtv9u3jjvavl6m8zcg9ifnwsuqirmqszbqiytoi4sjfdrnevvnf3tvazmlooxtbix9yfy+l056xjeosiayqw2corqyjqrangsgjojnbpajiqra7qxyum/xb0veknia0gnxahjiaw3v1+askmicabhrqgoqkdoicolt4hagdebrkjqfhtinsljewhbf6o1rjvhbkt/c9wojisrxgtmzlbof/kxbrdazwfz016jri+7ea8h1vtuyfavdu0/z2it/m1sqrnfas2x/5bux/+uwsbnj4tdmonkkhxk4nxjwotsv2hpybymcfbjml55jxiau5h3moynps7pzsoew/kdkynhzo28f32yvemfu9znwwh01zpgxph052nrqr3otbcafu430+gh14bnswa+g98t56n7zpvvhf+u/892fsdc95bsfvy//wa/ix6f8=</latexit> Recap: Pinhole camera The camera matrix can be decomposed as C = 0 1 0 1 0 1 r f 1 c 1 1 0 0 0 11 r 12 r 13 t 1 @ 0 f 2 c 2 A @ 0 1 0 0A Br 21 r 22 r 23 t 2 C @ r 0 0 1 0 0 1 0 31 r 32 r 33 t 3 A 0 0 0 1 Intrinsic camera calibration matrix 3 3 Normalized camera projection matrix 3 4 Extrinsic camera calibration matrix (a rigid transformation) 4 4!11
Recap: Pinhole camera In a more compact form: C = K [ R I t ] [ R I t ] is 3 4 K = intrinsic calibration matrix (3 3) Constant in this project Camera calibration R, t = extrinsic calibration The camera pose for each image in the sequence!12
Recap: Camera center The pinhole camera projects 3D point onto the image plane through its center n: The camera centre satisfies C n = 0 If C = K [ R I t ] n 2 R 4 <latexit sha1_base64="lqkmy+przn2or+9h/dbnijgtg8o=">aaac23icjvhlssnafd2nr1pfvcgnm2arxjvec7osunfzxt6grsvjp+3qvegmqqlduro3/obb/r/xd/qvvdnguivohcrnzr3nznx77ddlstcml4w2mzs3v5bdzc0tr6yu5dc3angqra6rooebra3bipnlfvyvxliseubm8myx1e3hiyzxr1gu88c/ekoqtt2r7/medyxbvce/nw7zpd2ftljf8iwxso3x+esy1mkxjkkhlj4nzbqukk5kkh9gc10ecjdaa4mpqdifhziejkwycilry0xcriirommeodimlmuowyj2sn8+7zop69neeszk7daplr0rkxxskiagviiwpe1x8uq5s/y377hylhcb0d9ovtxibqbe/qx7zpyvttyi0morqofttafizhvo6pkorsib61+qeuqqeidxl+iryucpp/usk02sape9tvt8vwvkvu6dndfbm7wlddj8oc5punsvmgdf86xukb+no85igzvyo3keooxtvfal72s84bfpwlu70w61u49ulznqnvftaffvjz+yjg==</latexit> The 3D coordinates of n is t = R T t Alternatively: if C = K [ R I R t ] The 3D coordinates of n is t!13
A practical issue The numerical values of a specific camera matrix always refer to a specific coordinate system in the image and in 3D space There may be more than one coordinate system that is used for particular problem: - A camera centred coordinate system - Another camera coordinate system - The world coordinate system We need to know which coordinate system it refers to in order to use it practically Maybe transform it to a standard coordinate system!14
Recap: Equivalent cameras Two cameras C 1 and C 2 with identical camera center n: C 1 n = C 2 n = 0 are called equivalent The images of two equivalent cameras can always be made identical by means of a homography transformation Disregarding that images are of finite size Cameras that are not equivalent are called distinct!15
Recap: Estimation Many problems where we estimate some type of geometric object can be formulated as A x = 0 x is a representation of the unknown geometric object that we want to determine Usually in terms of homogeneous coordinates Or, at least, in terms of a projective element A is data dependent!16
<latexit sha1_base64="wcqqhxgquu0oboxgvmadqmjhcv4=">aaadcnicjvhrthnbfd2silwavn30zwnjai/nrpbiykgqy8ijjrq06tbn7nrab7y7m51zu9l2d/gt33wjvpodpbnhb+qvvdmmidoqnc3unjn3ndnzz5i8fuohwy8f78hi0splyqpqyura4ye1p89aspyf400mu1m0k1jxvgs8qyvoetsvedxkun6unlw39appvfbczof6nofdutzmxecwwbpvq32iek5ekrp1szqm/pfsyyeawvhu5phogr2nnb/risqty860r6u/cypt3gl7txrqcozw50hoqb1uhmjad0toq4khxagcgtthfdeupr2ecjat18weuikqshwogarklunfsrete0lfic06js1objkvdtnajaw3ikepv+srpcsim9v8wy9tsmhvy57ytlo3u/onlmtermynyv/lu1p+r8/0ojhatu1bue+5zux3zkwu9ltmzv3futkukbnncj/qbwfmnxfn7fupsr2bs41t/advgtbmmdowudg7paso/77oedb63qg3g+hhrfrunrvqcl7gjdbppt9gf/s4qjoyv+asv7j2zryv3rl3csv1fpznof4y3rdfxk2r7w==</latexit> <latexit sha1_base64="krqnpcqrvactkbs7u8kgq4f5sy0=">aaac+xicjvhlshxbfd220ejex2iwbhohqugghhxius0m2yibzizgifsxnwnhv+iufqxph3hnlrj1b9yahxd/wpxfbl17icpiqumuc8+953tdun4s6mx43soim/phbpzjxgtt09t0zgx9br6txxkqvvvgqzzu+yjtgy5u22gtqp0kvsl0a9x1t7/afpdmpzmoo5/milghorheuq+lmeqd1teknt93t0vezstepsnekmyjfegxwy4z7zwrxa9cwdh3cuwo3vcahi/3nwhvoifq7cx1e/rwjbgsouiordceawhk9bygbq8jcycoiesjac4rlkirnqcqrrwc2fp6dig6qniiyuuzsvrsxwj6u1k6wcjnthupyfs3l/m5o1v2le+cpe3zlmj3k6+qwimtyt/tdsv/v2d7mehjk3vq1fpcjo1ovi4534o9uftpv4yceuispqz8sliycnjplmsy7t3ered8i1da1sayqs3xx56sbtx6oc7xolpwbk03135snlz2qlfpyaglwkz5fsewvmepbfk+xc3u8nspncvnl3p9voqmvjrpelacm7/3parl</latexit> Recap: Estimation In practice, the data in A includes noise The equation A x = 0 is not satisfied exactly Solution (homogeneous method): minimise: (x) =kaxk subject to kxk =1 This approach minimises an algebraic error Ax N (0, I) Optimal for Gaussian residuals, i.e. This is not the case in general!17
Recap: Estimation Algebraic errors lead to linear solution methods (e.g. using SVD), they are simple to use The actual data noise, however, occurs in the Euclidean geometric space (2D or 3D) Geometric errors are almost always non-linear functions of the free parameters that we optimise No closed form solutions exist Iterative minimisation methods must be used Good initial solutions are critical (use linear estimators!) Use optimisation tools, e.g., in scipy.optimize To be continued...!18
<latexit sha1_base64="0nyeo48sngvgsddyh/ikk3yu28y=">aaac4nicjvhlssnafd2nr1pfuzcibivgqiqq6llyjcsk9ggtlcsd1qf5kuzeurpy507c+gnu9wpep9c/8m6yglpejyq599x7zsyd60qet4rpvua0mdm5+yx8ymfpewv1tv/fqcdhgrus5ozegdcdo2eed1hncogxzhqz23c81nagfzlvxle44wfwloyru/dtfsb73lufur19e9r2eszw3lhacfdvukfagetxry+ajvmtyxpygsgiw9vqf0ebxyrwkcihqwbb2ionhj4wljiiilvailiyefd5hjekpe2pilgfteyavn2kwhkbucw9e6v2arep3piubnzje1jdtfjuzqh8qpwl+5v3shnksw3p72reprecl8t+pztu/lcnexho4vj1wkmnsdgyozdzsdwtyjmbx7os5barj3gx8jfhvykn92wotaj6l3drq/ybqpssjn2snsw7pcun2po5zmlq3y9zbyxr7lbypslgnccwdrbh8zxcgaeookben3jee561rnar3wn3n6valtns4tvshj4a+nya/g==</latexit> <latexit sha1_base64="afzbsltfhph3aswjshcvfz2zrzg=">aaac4nicjvhlssnafd2nr1pfvzcibivgqirv0gwxg5cv7apaupj0wofmrtirs+nknttx6w+41y8r/0d/wjtjcmornzdk3hpvotn3rh26pbag8zrr5uyxfpeyy7mv1bx1jfzmvj0okshhnsdwg6hpwzfzuc9qgguxncoiwz7tsoy9rmh844pfmq/8czekwcezbj7vc8csrhxzu+o23ddhk26phxnpbrukfliedpmfo2iopc8cmwufpksa5f/qrg8bhctwwobdehzhiaanbrmgqui6gbmxeeiqzzbbjrqjvtgqsigd0ndaustlfyqlz6zudu3i0hurusc+aqkqiwjl3xsvt5szzh/zhitpebyr/e3uyynw4jlyv3ttyv/qzc8cfzyohjj1fcpgdueklom6fxly/utxghxc4ituut4i7cjl9j51pylv7/julzv/u5wslbgt1iz4l6ekazs/xzkl6qwievg0z48k5dn01fnsya8hnm9jlhggkmrkfynhpofz62m32p12/1mqzvlnnr4t7eed/1ebaa==</latexit> Recap: Epipolar geometry A 3D point x is viewed by two distinct cameras C 1 and C 2 y 1 C 1 x y 2 C 2 x y 1 and y 2 are corresponding points Either: Two physically distinct cameras The same camera that is moving over time!19
<latexit sha1_base64="s693vokrtwomx8idxusclyid/ww=">aaac4nicjvhlssnafd2nr1pfvzcibivgqirv0gwxg5cv7apaupj0wofmrtirs+nknttx6w+41y8r/0d/wjtjcmornzdk3hpvotn3rh26pbag8zrr5uyxfpeyy7mv1bx1jfzmvj0okshhnsdwg6hpwzfzuc9qgguxncoiwz7tsoy9rmh844pfmq/8czekwcezbj7vc8csrhxzu+o23ddhk67zjrmnggofclxpuvmcuttu0mebmyic0lun8i9oo4cadhj4ypahcluwenptggkdixedjimlchgvz5ggr9qeqhhvwmqo6tugqjwypsxsm1zqh3zx6y1iqwofnahvrytlbrrkj8pzsr95j5wnpnui/nbq5rercensx7pp5x91shebpk5ud5x6chuju3nsl0tdijy5/qurqq4hcrl3kb8rdpryes+60ssqd3m3lsq/qurjythjaxo8y1psgm2f45wf9vlrpcywzo8k5dn01fnsya8hnm9jlhggkmrkfynhpofz62m32p12/1mqzvlnnr4t7eed+s6a/w==</latexit> <latexit sha1_base64="4vnogtkpb18uqo/uk+iioqta/9c=">aaac4nicjvhlssnafd2nr1pfuzcibivgqqrv0gwxg5cv7apaupj0wofmrtirs+nknttx6w+41y8r/0d/wjtjcmornzdk3hpvotn3rh26pbam+zrr5uyxfpeyy7mv1bx1dx1zqx4hseswmho4qds0rzi53gc1wyxlmmhelm92wcmevms+ccwimaf+hrifronza5/3uwmjorr67rht943rpftqx9xtqyucba4nxt1vfky1jflqteee6aog+gva6cgagwqeghwiwi4sxps0uisjklgoxsrfhljkm0yqi21cvywqlgkh9b1q1epzn2lpgsu1q7u49eaknlbpmodqisjyn0ple+us2d+8x8ptnm1efzv18ogvuct2l9208r862ytahyeqb049hyqr3tmps6jurz7c+nkviieqoil7li8io0o5vwddawlvu7xbs+xfvkvkzeyktqne5slpwmwf45wf9vkhefgonr/ly6fpqlpywr4oaj7hkommvdti+wapemkz1tnuttvt/rnuy6sabxxb2smh/6mbaq==</latexit> Recap: Epipolar geometry A 3D point x is viewed by two distinct cameras C 1 and C 2 y 1 C 1 x Epipolar plane L 1 y 2 C 2 x x L 2 l 2 y 1 y 2 l 1 n 1 e e 12 21 n 2 Camera 1 Camera 2!20
Recap: Epipolar points The image of camera center n 1 in camera 2 is called the epipole (or epipolar point) e 21 : e 21 = C 2 n 1 Correspondingly, the image of n 2 in camera 1 defines the epipole e 12 : e 12 = C 1 n 2!21
<latexit sha1_base64="eq/xv1ypjyyvg6s0d7wfft4fpqi=">aaac4nicjvhlssnafd2nr1pfuzcibivgqirv0i1qfmrlhb6grsvjpzu0l5kjuepx7tyjw3/arx6m+af6f94zu1cl6iqkz86958zce63qdwku668zzwz2bn4hu5hbwl5zxvpxn2pxkeq2q9qbg0qny4yz6/isyh3uskyymdozxfa3bqcixr9muewefoupq9b2zl7v9bzb5er11o1ry+ppw3hhukxiedaemmvjvapm9yiulzynjbtkka5yol6ghs4c2ejggcehj+zcrexpewz0hms1msiuiutiommyodimlmuowyr2qn8+7zop69neemzsbdmplr0rktxskiagviiwoe2t8uq6c/y375h0fhcb0t9kvtxioa6i/us3yfyvtttc0corrmghmkljiors1cwrxre3175uxckhje7glsujwrzutvqssu0saxe9nwx8twykvuztndfbu7glddj4oc5pucswjp2ccxgql52ko85iczvyo3keoorzlfel7xs84gnpsle5ve6u+89ujznqnvftkq8fxakafq==</latexit> Recap: Epipolar constraint In this case, it follows that there exists a 3 3 matrix F, the fundamental matrix, such that y T 1 Fy 2 =0 The epipolar constraint between corresponding image points y 1 and y 2 F depends only on C 1 and C 2 : F =[e 12 ] C 1 C + 2 <latexit sha1_base64="sjbp+7+ferlm8hyibkiwdw8bbdg=">aaac+hicjvhlssnafd2n73fvpztgeqshjk2gg0euxkwc1ujbqxkndwhejbnbqz7enttx6w+41s8q/0d/wjtjcj4qnzdkzln3njl7rxn5pbgg8vlshozhrsfgjyanpmdm58rzc8djmmyua7ihf8znx06yxwpwefx4rbnfzpydj504/v0zp7lgccld4ehcrqzj272ad7lrc6kscj1ro119l99qkcbykznrecdqc+6zrhg7uwuoqo00w8utcswogmrpp4fzgaqkdrcwn9hgguk4sogdiyag7mfgqk8ljgxexhwqerct4irokgostcllmcqwie3tt0e7vsegtjeeivk7diphb0xkhsukcskvjixp01u8vc6s/c07u57ybpf0dwovn1ibc2l/0g0y/6uttqh0salq4frtpbhznvu4pkor8ub6p6oeoutesxxg8ziwq5sdputkk6jazw9tfx9vmzkve7fitfemb0kdnr+p8yc4rlxnetu8xk9s7xsjhscslrfk89zanvzxgaz5x+mbj3jsrrqb7va7+0jvsovmev+wdv8omwcjfa==</latexit> Pseudoinverse Calibrated case: F is determined from known C 1 and C 2!22
Recap: epipolar lines F maps y 2 to a line l 1 is an epipolar line: it goes through the epipole, If y 1 corresponds to y 2 : then y 1 lies on l 1 : l 1 = Fy <latexit sha1_base64="h7ja8wawoz6vpwgtms+ptt2czho=">aaac4xicjvhlssnafd2nr1pfvze6cbbbvumqobuhkijlcvybbsnjdfpd0yqke6gubny5e7f+gfv9gfep9c+8m6agftejmtlz7j1n5s61a9ejhgg8zrsz2bn5hexibml5zxutv75ri/w4zlzkfncpg7yvcdfxefu4wuwniotw0hz53r6cynj9moer43uxyhtw9tdqe07pyzygqppfhrfsnu4mnbgzhct8lqhllhrknxzbkbpq6npateeb6aj4+re00iuphhhdchgqhf1yiohrwosbglg2xssfhbwv50iqi21mwzwylgihnpdp10xzj/bsm1jqrqe49iek1lflgp/yqslynf3fy+us2d+8x8pt3m1eq516dykvucl2l90k8786wytad0eqbodqchqjq2ops6xerd5c/1kviieaoim7fa8jm6wcvlounjgqxb6tpejvkloycs/s3bjv8pbuypnno6dbrvq094vmxughfjk2oost7gcp+nmims5rqzw8b/cijzxrtlvv7rt7z1qtk2o28w1odx+s6jp8</latexit> 2 y 1 l 1 =0 <latexit sha1_base64="uxuet+o5niqxqc8crruxsrihe2u=">aaac4hicjvhlssnafd2nr1pfvzfdbivgqiqq6eyounfzwt6glsgztmtomorkiptqhtt34tyfcktfi/6b/ov3xhtuijohybnn3nnm7lwn9nxygmzrtpubx1hcyi8xvlbx1jekm1unoegixuss8iko5dgx91yf14urpn4ki26phi83negzzdevers7gx8pxihvjuyb7/zdzguirgip7th9ftyxunpsyb1aqnibwoajyrxlrsvqs58fzgbkyfytkl6ggx4cmcqygcohiozbrkxpgyymhmr1krixexjvnmocamktqujuyrm7po+aonbg+hrlz1ipge3i0ruruscuaqkqiwjl3xsvt5szzh/ztpwnpnuy/k7mnsjw4iryv3ttyv/qzc8cfryrhlzqkvsm7i5llom6fxly/utxghxc4ituut4izjryes+60ssqd3m3tsq/qurjyphltqne5slpwobpcc6cxn7fpkiyf4fl6mk26jxk2meezfmivzyjhjp53+art3jwho1wu9pup0u1xkbzxrelpxwayw+ajq==</latexit> in the first view e 12 l 1 =0 <latexit sha1_base64="bsgdmwcvupdgdnbhoiwba10mzro=">aaac4xicjvhlssnafd2nr1pfvze6cbbbvulu0i1qdooygn1aw0iyndbqnanjrcihg3fuxk0/4fz/rvwd/qvvjcmorxrckjpn3nnm7r1o6lmxmizxndyzoze/kf8slc2vrk4v1zfqczbejndy4avr07fj7rk+rwlxelwzrtweoh5voimzgw9c8yh2a/9sjelegdp93+25zbzewcxtto30dd62unn/3gbdqcjcg1vmiwevs0bzueufbmygsshwnsi+oi0uajakgildhydswuzmtwsmditedzasfxfyvzxjjajpe8rilgeto6bvn3atjpvplz1jpwz0ikdvreodu6qjkc8ile/tvtxrzpl9zttvnvjui/o7mdeqwieryv/sttl/q5o1cprwrgpwqazqmbi6lrkkqivy5vqxqgq5hmrj3kv4rjgp5atputleqnbzw1vf31smzowezbkj3uutacdmz3fog/p+2twomxehpcppnuo8trcdpzrneso4rxu18r7bi57wrdhtvrvt7j9ttvym2cs3pt18ad3lmk0=</latexit> Similarly for the other epipolar line l 2 = F T y <latexit sha1_base64="unnc4neirllcfslmsirl1rppogk=">aaac43icjvhlssnafd2nr1pfuzecbivgqirv0i1qfmrlhb6grsvjpzu0tuiyeurjzp07cespunv/ef9a/8i7ywpqez2q5my595yze68vue7edf01o8zmzs0vzbdzs8srq2vq+kyt8upqzlxbd/2wyzkrcx2pvbndxdyiqmyolzfvrcgpinevwrg5vlfho4c1h2bfc3qobxkiour2ugx1ndfpjivjscrnywvfglhsmtpqxi/ocmntwehbhukq++olwujch40yqzb44irdmijoackajoc4nsbehyqcgwdikcnttfmmmkxib/tt066zsh7thwck1tad4tibklldlml8ygsji9m0gy+ls2b/8x5lt3g3ef2t1gtilmcvsx/pjpn/1ylaoho4kju4vfmggvgdnbresivi5tqxqjg5bmqj3kv4sniwykmfnamjzo2it6amv8lmwyq9nebgebe3paebp8c5dwrfgrffmc4o8qwtdnrzbgeheztpq5rwjjkq5h2drzzhwwhkrxkn3h+mkplus4lvs3n4alsem0i=</latexit> 1!23
Recap: 8-point algorithm F can also easily be estimated from a set of 8 (or more) corresponding image points: Form a data matrix A Determine right null vector f of A (SVD) Reshape f to F Enforce internal constraint det F = 0 Important to use Hartley normalisation to increase accuracy!!24
Recap: triangulation Given two corresponding image points y 1 and y 2 we want to determine the 3D point x Mid-point method Algebraic method Probabilistic method ( optimal triangulation) In general, different methods give slightly different results if y 1 and y 2 are noisy!25
BREAK!26
<latexit sha1_base64="rrbfacsdjob2rb5n9m31cod+a18=">aaac4xicjvhlssnafd3gv31xxeoiwiqkupiq6lloxpuowbwslmk4byfmrtirpbhx507c+gnu9wfep9c/8m6ygg9ejyq5c+49z+be68w+tjxjvaxyg0pdi6ofsfgjyanpmels3geazqkxdr75uxlssvt4mhr1jzuvjunesmdzxzhx3dbxo3orpdikd9rlle4d1g5ls3kmigowfy8aqqwaavmdzvze7lw5ewsrxludttislpyky5b9e7g5kcffe1hxgq2ciqjhhgacirrhhwwppsdw4sam7hq94hjc0sqfrjbo2oyybguwyrv0bdpujgdd2mvp1kg5neltm5dsxjjpisplcovtbbppjlnmf/pugu99t0v6e7lxqkxch9i/dp3m/+p0lqotbjoajnuug0zxx3oxzhrf39z+vjuih5g4jc8onhdmrtnvs200qald95az+kvj1kze8zw3w5u+jq3y/t7on+cwwnhxktx99vjtkx91aqtyqpnmuyeadrchonlf4wgpelk4dwpdwncfqdzarpnhl2xdvwmerjsj</latexit> <latexit sha1_base64="warpv8c6mdhdkhu6ua3lcsn3ynu=">aaaddxicjvfntxrbeh2miaikqxy5tnyqlimsmynglyybuexxtvwgyvjs0/suheal7h4dgfc3+e+4etne/qmmxooe9v9q3qwejuz7mjovx9v73vwvlknujgi+zxg3zm/o3zq/vbb45+7svdb9b1u6qbqxq16khdpjmbapzmxqsjoknvijliwp2e4on218+61qwhb5a3nsir2mtxi5lpwzovzb/yhqhl+ls+prrouky2sv4rfiva6ndayplkmjujtnove0nyprvr50n3ao10mynooex4b2w+2gg7jlxwdha9po1qbofuwmaxtgqjbbiichnijb07oleafk4vzqe6cisrcxmgkbtbvlccpgxb7sd0k73ybnaw89tvnzoiwlv5hsxyppcsptho1pvotxztmyf/ounae92wn9k8yri9bgdbh/0l1k/q/o1miwxnnxg6sassfy6njjurmu2jv7v6oy5fasz/ebxrvh7pqxffadrrvabw+zi/90mza1e97kvvhlb0kddv8c53wwfxxdj93o1ua797iz9txw8badmucz9ndhaepypsuxfmcp7733wfvoftpp9wyaztj+w97nmz3crce=</latexit> Maximum Likelihood For normally distributed noise we can often define a maximum likelihood solution to estimation problems Consider a direct observation x: x N (µ, ) with the observation likelihood: 1 Pr(x µ, )= p e 0.5(x µ)2 / 2 2 2!27
<latexit sha1_base64="lcwdi2rr4dsgng074ajb2yomde4=">aaadkxicjvhptxqxgh0mkoi/vjh6mbgxwcrsztbeliqel5zimrbawucm0ylrq2c6atvgzcpfxh/ijrpklyvevoljr2vivonrtqbz+r733vrr81pj65lk60w0e+v2nbn5uwv37j94+kjzehhh6szwmebaaboxmyuurmtasafexm0ek3mldvojn76++0eyk3w17ca1ocjzqjkhkjnh1lbt9m3yx2h6ksubf5mvo5i9z3ihxazosghgqbi1lvnlcmlrxcbwjzwis9royjip1tljd1tx8mpapux0k14srnwtpc3ooh193tldhgiaha1kcfrwhbuyld37sjggju4ae+imirnqasdyig9dkkekruwrzsna7bdsrwufayob018uvyacmz6rr5poepz/i0o9ccme/vv2jgt6vy3pm7dzjbeo74n9l+9a+b8+34vdivzdd5j6qgpju+ntshnoxe88nurkuujnnmcf1q1hhpzx5xwhjw29+7nlof4tkd3r17zvnvjud0kxnp55ntfbzkovfdlbefuqu77rxvu8nuapluk+x2mdm+hjqnmf8qox+bmdrkfrl+j8shrntj4l/daii19mklha</latexit> <latexit sha1_base64="nbgmkwlsdh6xwma2+uqb2cg8pgm=">aaac7nicjvhlshxbfd3tgumrourstemgkaxdtwrjuntjshqcz8cwobqmhaurh3rvb2xwg9y5k2zza27nzwt/qp/cw2uljkoi1xt3qxpvovx33jhxupsgekx5y+ofjj5ptk3pzh6zm68vlb7rrcy4apnmzuu3zloomyq2kuajbl4ilsrkdoklxrvvfbeflll6zk5yczqwqsrpjgegqf59/bixnipvz4xuxvb2iy2tkghmndm13l9ei5kysdwgyeu9einobw75oycsqapvosjqvxghjwwcjriipdcefrg0psciesan7hrd4gpc0sufrjfn2pkybguwyi/oo6ddscwmtlee2qk5nalolujpy5u0geuvho1pvouxztmy//ieok97tyv6x5vxqqzbobh/071lflrnaze4wzdxg6sacsfy6njlurqu2jv776oy5jatz3gf4gvh7prvffadrrvabw+ziz+5tmvapa9yszzbw9kaw7/hoqqon1rhzmvjckuxvvonehllwmeazfmrtrgha7tj+wb3emavl/duvtvvx2uqv6s0s/hjet9fagqfobo=</latexit> Maximum Likelihood If we have several independent observations: x 1,...,x N N (µ, ) Their joint likelihood becomes: Pr(x 1 µ, )... Pr(x N µ, )= NY n=1 Pr(x n µ, ) We can now consider the estimation problem as one of finding the parameters that maximise this joint likelihood.!28
<latexit sha1_base64="rs0ihqoildahisgefi7gyjy9iq4=">aaadghicjvhlssqwfl3w93vupzviikji0kqgg0f040pgdfqwtkk7mu6wl9julhf+xd9x507cuhfxbvuvvikrx4imtd0595yt3mtpip5lx3nssrp7evv6bwahhkdgx8zle5mhevqignwcnerfku9zfvge1ssxetvkbkoxh7fd/3rl1q/pmmh5muzli4ydxdrmejmhvclllfzixjqstelc1bcwsc7dmnyxiobhs5looqeijde995taldske2ndayfrbqe+y1ff3lmxxh7u571s2ak4etg/gwtagcyopquhincafaioiaygcujeevdi8tkgfxzikdubnnicedd1bh0yqm+bkoykiuwpfkochrs2wbnkzlu7wfuifau6bzhft4o6gvitzut6ozmv+1t2w2eqvv3g3zdzmbiswsj+5xtx/tenephqhdxda8eems2o7gktuuhtutu3p3ulmsfdtueg1gxiqdvfz9nwnlz3rs6w6vqzvipwzqojlebf7riv2p1+nt/bwvlfxa4s7a6unzbnvq/anmzahn7nkmzanlshhtnx8itrvvhx1o11a929sa0u45mcl8o6fwxv37hk</latexit> <latexit sha1_base64="wpvtyqhmtu/c3zu9mx3mydk7foq=">aaadbxicjvhlshxbfd12ho6tgmdk6azxckji0d0j6eyykk3iagkocpyo1w1nw0y/qk4whmhw+zpssgvz+gnuk5a/0l/irupjhijatxefe+49p+rwjcpuvjoifs55dx4+ejzfwgg+ebr4bkm1/hyvkmovi0fcpiu6ihglupmlgzy6fqelejylurefjd+a/p6zujus8l09kcvrxpncjmtmnvhdvo+lyqtzlgx18cymq2ss8emnpmryqtlsh3gvsezmq1pg0rp3a3+c9wgrhxqcu/ybihsgdbf6reshge5qieandai5noeuhbu9hwgroctucfpifcfp8wiznelbu5wgck7sml4jryeozsk2npvvx7rlsq8ipy+xpcmothe2u/k2x1tnw97mpbwe5mwt+kfokynw45tyu3txlffvmv40rti2pujqqbsm6s52lrw9fxny/6+undmuxbl8qnlfolbk63v2raayvzu75tz/assna+ly1da4mqekayf/j/mm2ot2wled7ofx7d4bn+ogvrcknzrnfnp4hz4g5p0jf/ig795h77p3xfv6u9sbc5ox+gd5578arimp2w==</latexit> <latexit sha1_base64="kdzafjo7etsb/3kpeiri5nume28=">aaadj3icjvflb9qwgjygvymvby5cilziuxjdkobel5uqucaoqehsw6nprhzxu7xqpgq7vvfr/id+cbfechy5wyurhpnszqvchcbrkvf8m2n/dlypawwufv4kll2+cvxa8vwvgzdv3b7tuxtvy5s15mlis1xqnywzowqhhlzajxyqlviekbgdhb509e0joy0si3d2vom9ne0lozgcwalgnex1l83rx6mr05z119ojzryj500yt02dj5tipz6p3os943gxssl+khlyph0lq6kvx9bymrlnbaqc9pk0kgtzf9zpropij/aiifvqrts2y85hpnhhcy4aoqqkwmikdiaexcsiubg3h4y4tuj6usack+stssviwyg9po+uzrstw9dczrrv5rskoletm8qj8psk04tdaqgv1z7zsx/lbnym29um/lmblrnrcudsv3wl5f/6xc8we6z5hit1vhngdcfblnqfitt5ek4rswkvcq7vu10t5t65oofqe4zv3z0t8/wvxulyn+ettsy3t0u64pjp67witpjb/hsqvh3w3xjrxvuyhuahensfz7gbv9jekli/4dt+4gfwpjgjtonpz9jgqfxcx28j+pilax62ia==</latexit> Maximum Likelihood Maximise the joint likelihood: Take the negative log of this expression: For: {µ, {µ, } = arg max µ, J(µ, )= 1 2 } = arg min µ, NY n=1 Pr(x n µ, ) J(µ, ) NX (x n µ) 2 / 2 1 2 log(2 2 ) n=1!29
<latexit sha1_base64="klkxulstgogclr6sas/radeqhho=">aaadanicjvhltttafd0xpyrhs4alg6srekhv5aasbjaqbbcruduecsm0nkzccl8yj0frlb1/wo4dysspsiuf4g/av+idqamwufxgsn3m3hvozl03seozac97ljkjh0y/jpxhjyanpn2erszm7mdjrrho8irm1ehamhhkwds11ke4sjvgurckvnc6bektc6eymctfds8vrxhrxrijodnehvc2/i5ivo+ntgnjqnd30y/yr34muxfbgvzhir1sep5zztr+n9k90uyp5ok4uvvqnl3uw1avqbxfaisvj/hoiwfhjggcmttheawzpyeow0nk3bh6xclc0syfbpggbu5zgjiysaf07dlusgbj2hvpzko5nrlsq0jpyoe0ceupwuy018zz62zyf3n3rae5w4/+qeeveatxquz/dmpm9+pmlrodrnsajnwuwszuxwux3hbf3nz9qypndilxbrcprghzqxz22bwaznzuests/ifnnkzz8yi3x09zsxpw/fu434l95vp9pba8t1rd3cpgxcy8vmcr5rmgteyggsz5x+eed3h0lp1r58a5/z3qlarnhf4s5+4xepco9q==</latexit> <latexit sha1_base64="01rvdcjblztkdrh9wp+yxfktsd4=">aaadbxicjvhlsurafd1mfd97xqwbycmosjnwqtdcm27elso2ckakul3dfp2xlyqdnl2ep3hnttz6a2518a/0l7xvrnwhwihjqxpvovx33ianzay9767h+dhb1z8wodq8mjo2plh6+wsns3lfrz0nyal2apajumairquoxv6qbiucuowg7tut3z0rkpnjvk1pu3eqsvysm5iztdrhqey3femdp2vksxa6g7n+lm/7mwxfbk77wj8x3vxpp85zw9+srspnler+hpbkxswzy/0iqguoo1ibsek/fdsqgcnhbieymnaihoyefvthisxuab3ifcfp4wjddjm2pyxbgyzynn1btnsv2jj2xjozak6nhpqqurqyiu1ceyqwoc218dw6g/yz7471nhc7px9qeexeahwr+5xuofo7ololrhmrtgzjnawwmdxxwiw3xte3d19vpckhjc7gbsuvyw6vz312rsaztzvemhu/t5mgnxte5oz4mlekavffj/mj2fmovbcrc3+wyrxfxaghmyvpzni8l1hdojzrj+8zxomgt84/59y5cc6fup2eqjojn8u5egsc26pc</latexit> <latexit sha1_base64="3zevjr+4jbji4wdjx+cdbv/izss=">aaac8hicjvhlshxbfd12nde+4iqu3tqoasvk6b4dyuyq3bgsbucf2xmqy5qxsf9uvydimx+rnttxmx/ivr8i+af6f94qw/bb0gq6+9s595yqe29cjfkbilga8d6mjr19n/5+ynjq+snm4+onhz2xiosoz5nc7cvmi0rmomokscreoqrl40tsxsdrnr77uygt82zbnbtiigwdtpylz4aoxunlpougzd32ctrxjffhsnoyrrpme1w2ha67gwu/evjrljal3xav0qxagvv+cxdwoil6beanf4hwibwcjviizdceezboevyrikbb3aeq4hqh6eicq0yqtqqsqrmm2gp6dmi3x7mz7a2ndmpopyt0kll6+eyanpiuyxua7+klc7bs/7wr52nvdkl/upzkitu4ivyl3x3ma3w2fom+frgajnvuomzwx2ux0nxf3tx/ujuhh4i4iw8prghzp7zvs+802tvue8tc/nplwtbuez1b4sbekgycph3nc7dtbovlrfbwt+bkaj3qccxhhgs0z+9ywto20shv3/ilc1x6yjv1zrzzu1rvpnbm4thy/twc1o2gmw==</latexit> <latexit sha1_base64="g7rpt4x/h2jjifnkekatw0mdfig=">aaac43icjvhlssnafd3gd31vxqoslikrkqigm4loxpuo2fawwpjxqon5mzmierpz507c+gnu9v/ep9c/8m6ygg9ejyq5c+49z+be6yebsjxjvprz/qodq8mjo6wx8ynjqfl0tconm8l4ncvbla98l+wbihhdcrxwg0ryl/qd3vtpt3s8ecflkujox10l/cj0tiprecxtrlxl860wq7u60mo52813uq00c9t5vho7xzuxbdeuv5yqy5b9e7gfqkbyu3h5gs2ciazdhhacerthab5seg7hwkfc3bfy4iqhyeicxzrim1ewpwyp2hp6ntlusgaj2mvp1kgznrlqk0lpy5e0mevjwvo028qz46zz37xz46nvdkv/v/akivu4i/yvxs/zvzpdi0ih66ygqtulhthvscilm13rn7c/vaxiisfo4xoks8lmkht9to0mnbxr3nom/moynav3rmjn8kzvsqn2v4/zj2gsv92v6vleamvjsxj1coawgcwa5xo2si1d1mn7gg94xjpfrrvr1rr7slx6cs0svizr/h2ghzwo</latexit> Maximum Likelihood Setting partial derivatives to zero gives us estimates: and @J(µ, ) @µ @J(µ, ) @ =0 ) =0 ) µ = 1 N 2 = 1 N NX n=1 x i NX (x i µ) 2 n=1 True μ!!30
<latexit sha1_base64="s693vokrtwomx8idxusclyid/ww=">aaac4nicjvhlssnafd2nr1pfvzcibivgqirv0gwxg5cv7apaupj0wofmrtirs+nknttx6w+41y8r/0d/wjtjcmornzdk3hpvotn3rh26pbag8zrr5uyxfpeyy7mv1bx1jfzmvj0okshhnsdwg6hpwzfzuc9qgguxncoiwz7tsoy9rmh844pfmq/8czekwcezbj7vc8csrhxzu+o23ddhk67zjrmnggofclxpuvmcuttu0mebmyic0lun8i9oo4cadhj4ypahcluwenptggkdixedjimlchgvz5ggr9qeqhhvwmqo6tugqjwypsxsm1zqh3zx6y1iqwofnahvrytlbrrkj8pzsr95j5wnpnui/nbq5rercensx7pp5x91shebpk5ud5x6chuju3nsl0tdijy5/qurqq4hcrl3kb8rdpryes+60ssqd3m3lsq/qurjythjaxo8y1psgm2f45wf9vlrpcywzo8k5dn01fnsya8hnm9jlhggkmrkfynhpofz62m32p12/1mqzvlnnr4t7eed+s6a/w==</latexit> <latexit sha1_base64="4vnogtkpb18uqo/uk+iioqta/9c=">aaac4nicjvhlssnafd2nr1pfuzcibivgqqrv0gwxg5cv7apaupj0wofmrtirs+nknttx6w+41y8r/0d/wjtjcmornzdk3hpvotn3rh26pbam+zrr5uyxfpeyy7mv1bx1dx1zqx4hseswmho4qds0rzi53gc1wyxlmmhelm92wcmevms+ccwimaf+hrifronza5/3uwmjorr67rht943rpftqx9xtqyucba4nxt1vfky1jflqteee6aog+gva6cgagwqeghwiwi4sxps0uisjklgoxsrfhljkm0yqi21cvywqlgkh9b1q1epzn2lpgsu1q7u49eaknlbpmodqisjyn0ple+us2d+8x8ptnm1efzv18ogvuct2l9208r862ytahyeqb049hyqr3tmps6jurz7c+nkviieqoil7li8io0o5vwddawlvu7xbs+xfvkvkzeyktqne5slpwmwf45wf9vkhefgonr/ly6fpqlpywr4oaj7hkommvdti+wapemkz1tnuttvt/rnuy6sabxxb2smh/6mbaq==</latexit> Optimal triangulation Given two cameras and two observations of the same 3D point: and y 2 C 2 x y 1 C 1 x Epipolar plane L 1 x L 2 l 2 y 1 y 2 l 1 n 1 e e 12 21 n 2 Camera 1 Camera 2!31
<latexit sha1_base64="kgdzxxlljmxiy1uvctujw7adryy=">aaadhnicjvhntttagjy4uekanqvhlhzrjxqj7fcpvvskmkupqur+eizobtbbwn9arxfrmnfhtxrrdxfseqakumar8o2yqvce6fq2z2e+md1v18+imjeoc16yxs3nv14ol1awllfevk2+w+3masec3gnskbv9n+u8chpekagmed8tnmv+xhv+yuvpvsmu8jbntuq447sxgyxhmayyjgpq3z54/ta+nu55ljdfpszgxsyob4ztiw2nxtob+1ojfiejfnygnu7vxkwdvgto3dhdfgpca2owo51w/8ddpliekbcdi4ekhiehp2chlhxkxo1iqpwgfgqdy4okequq4ltbid2k74hmo4znak4yc+0oajwixkfogx/ik1kdikxws7ve6gtfppc90zlqb2p6+yyrjlbignixflpk//wpxisg+kj7ckmntdoqu8ckfppu1m7tb11jssiiu3ifdee40m7zodvak+ve1dkyrv/qsswqewbqc1ypxdifu/9e51pqbdtdzxrjx6da85u56jlwsi4nus/paoi72uhq9i9c4bo31on12zq1zu5krzlxvmejyf29bcehsxg=</latexit> <latexit sha1_base64="s693vokrtwomx8idxusclyid/ww=">aaac4nicjvhlssnafd2nr1pfvzcibivgqirv0gwxg5cv7apaupj0wofmrtirs+nknttx6w+41y8r/0d/wjtjcmornzdk3hpvotn3rh26pbag8zrr5uyxfpeyy7mv1bx1jfzmvj0okshhnsdwg6hpwzfzuc9qgguxncoiwz7tsoy9rmh844pfmq/8czekwcezbj7vc8csrhxzu+o23ddhk67zjrmnggofclxpuvmcuttu0mebmyic0lun8i9oo4cadhj4ypahcluwenptggkdixedjimlchgvz5ggr9qeqhhvwmqo6tugqjwypsxsm1zqh3zx6y1iqwofnahvrytlbrrkj8pzsr95j5wnpnui/nbq5rercensx7pp5x91shebpk5ud5x6chuju3nsl0tdijy5/qurqq4hcrl3kb8rdpryes+60ssqd3m3lsq/qurjythjaxo8y1psgm2f45wf9vlrpcywzo8k5dn01fnsya8hnm9jlhggkmrkfynhpofz62m32p12/1mqzvlnnr4t7eed+s6a/w==</latexit> <latexit sha1_base64="4vnogtkpb18uqo/uk+iioqta/9c=">aaac4nicjvhlssnafd2nr1pfuzcibivgqqrv0gwxg5cv7apaupj0wofmrtirs+nknttx6w+41y8r/0d/wjtjcmornzdk3hpvotn3rh26pbam+zrr5uyxfpeyy7mv1bx1dx1zqx4hseswmho4qds0rzi53gc1wyxlmmhelm92wcmevms+ccwimaf+hrifronza5/3uwmjorr67rht943rpftqx9xtqyucba4nxt1vfky1jflqteee6aog+gva6cgagwqeghwiwi4sxps0uisjklgoxsrfhljkm0yqi21cvywqlgkh9b1q1epzn2lpgsu1q7u49eaknlbpmodqisjyn0ple+us2d+8x8ptnm1efzv18ogvuct2l9208r862ytahyeqb049hyqr3tmps6jurz7c+nkviieqoil7li8io0o5vwddawlvu7xbs+xfvkvkzeyktqne5slpwmwf45wf9vkhefgonr/ly6fpqlpywr4oaj7hkommvdti+wapemkz1tnuttvt/rnuy6sabxxb2smh/6mbaq==</latexit> Optimal triangulation Given two cameras and two observations of the same 3D point: and y 2 C 2 x y 1 C 1 x Gives us the joint likelihood to maximize: x = arg max Pr(y 1 C 1 x)pr(y 2 C 2 x) x Negative log gives us a least squares problem: x = arg min d 2 (y 1, C 1 x)+d 2 (y 2, C 2 x) <latexit sha1_base64="hfiritf0mdkdbbnied+ltodtwdw=">aaadixicjvhlssnafl3gv31xxbojfqgilcqkuhheblxwsfuwbzmk0zqyf5ojweq/xj9x507cipgdutqv8m44fbwitkhy5px7zsyd8zkapckyhkem0bhxicnc1ptm7nz8qn5xqzbggfdp1y+dmj96jkubi2hvmbhq04rtenobpfeuyli/uaq8zxf0llojryeke7e284laqpl3e67xnq/6dzekys8lvooglgpqttvwigp2+017u6eyiq2ub3zvnu/dgejnfmeqwwqyw8dwoab6vol8a7jqghh8ycaecheixaeqspe5axsssjcrqw85jogpnuifptgbyrxfcolsbx47odvtbirzmzkqt4+rbphydjqwhp4y6zhiuzqp9ewls/a37j7klhvr4t/twsgyas6r/cs3qpyvt/yioa27qgegpswkkd35oivtpyj3bn7psmbcgpzeldq5yl85b+dskk+qepdns5t+rcolk+e+rs3gre4sl9j+ez3dooau7k2sc7rd2d/qv52dfvifit7nduzdivsgitk3unorvbnxxq1xz9x/lboj2rmm34bx9a5blrmj</latexit> x!32
<latexit sha1_base64="hfiritf0mdkdbbnied+ltodtwdw=">aaadixicjvhlssnafl3gv31xxbojfqgilcqkuhheblxwsfuwbzmk0zqyf5ojweq/xj9x507cipgdutqv8m44fbwitkhy5px7zsyd8zkapckyhkem0bhxicnc1ptm7nz8qn5xqzbggfdp1y+dmj96jkubi2hvmbhq04rtenobpfeuyli/uaq8zxf0llojryeke7e284laqpl3e67xnq/6dzekys8lvooglgpqttvwigp2+017u6eyiq2ub3zvnu/dgejnfmeqwwqyw8dwoab6vol8a7jqghh8ycaecheixaeqspe5axsssjcrqw85jogpnuifptgbyrxfcolsbx47odvtbirzmzkqt4+rbphydjqwhp4y6zhiuzqp9ewls/a37j7klhvr4t/twsgyas6r/cs3qpyvt/yioa27qgegpswkkd35oivtpyj3bn7psmbcgpzeldq5yl85b+dskk+qepdns5t+rcolk+e+rs3gre4sl9j+ez3dooau7k2sc7rd2d/qv52dfvifit7nduzdivsgitk3unorvbnxxq1xz9x/lboj2rmm34bx9a5blrmj</latexit> Optimal triangulation A least squares problem: x = arg min d 2 (y 1, C 1 x)+d 2 (y 2, C 2 x) x Each 3D point x defines an epipolar plane, which defines two epipolar lines. We can look for an epipolar line in one image, and transfer it to the other image. This leads to a 6th degree polynomial. For the best line pair, find the closest auxiliary points (y1 =C1x and y2 =C2x) Triangulate these, using linear triangulation (OK as these satisfy the EG perfectly, and have zero reprojection error)!33
Optimal triangulation You will use optimal triangulation in CE3. Source code is provided. Full algorithm is described in IREG 16.3 More details on the derivation are given there.!34
Robust estimation We often have two categories of errors: Measurement noise: we cannot determine numerical values of the data with high accuracy Outliers: all measurements in the dataset cannot be fitted to a consistent model We refer to data that can be fitted to the model as inliers Robust estimation: to determine a model from a dataset that contains significant amount of outliers!35
A chicken and egg problem We need corresponding points to estimate F Point correspondences can be verified if we know F Can we determine F and verify correspondences at the same time?!36
Robust estimation Model estimation from measured data where some of the data items are incorrect. E.g. Data Points in 2D or in 3D Lines in 2D or planes in 3D Model A 2D line, A 3D plane Corresponding points in 2 views Homography Point of intersection in 2D or 3D Corresponding points in 2 views Fundamental matrix!37
Example: estimation of a line from points!38
Example: estimation of a line from points!39
Example: estimation of a line from points!40
Observations We need (in this case!) a minimum of 2 points to determine a line Given such a line l, we can determine how well any other point y fits the line l For example: distance between y and l If we pick 2 random points from the dataset: We can easily determine a line l l is the correct line with some probability p LINE p LINE is related to the chance of picking only inliers p LINE is larger the fewer points that are used to determine l In general: if l is supported by the data there are more additional points that lie on it.!41
Probabilities Let S be a set of S =M points in total and M 0 of them are inliers There are M (M - 1) ways to draw 2 distinct points in a certain order, ( M 2 if M is large) This means p LINE p 2 POINT = (M 0 /M)2 This is in practice an upper bound: Even if we pick two inliers, the corresponding line may not be supported by the remaining inliers!42
Line estimated from 2 inliers!43
Basic iteration 1. Draw 2 random points from S 2. Fit a line l to the points 3. Determine how many other points² in S that support the line l within some error bound. ² These form the consensus set C 4. If C is sufficiently large, then the found line is probably OK. Keep it!44
Basic algorithm Iterate r times 1. Draw 2 random points from S 2. Fit a line l to the points 3. Form the consensus set C, together and Count the number of points in C (or p(s l) average likelihood of the dataset given the line) 4. If the consensus set is sufficiently large, then the found line is OK. In particular, keep l if N or p(s l) is the best one this far. Each iteration increases p SUCCESS = the probability that the correct line has been found We need to iterate sufficiently many time to raise p SUCCESS to a useful level!45
RANSAC This algorithm is called RANSAC RANdom SAmple Consensus Published by Fischler & Bolles in 1981 "Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography". Comm. of the ACM 24: 381 395. Several extensions / variations in the literature Preemptive RANSAC, MLESAC, PROSAC,!46
RANSAC for F For estimation of the fundamental matrix F, points that satisfy an epipolar constraint for some F are assumed to be inliers The points that violate the epipolar constraint are outliers!47
Chicken and egg revisited Let there be two views with a point set P 1 in one view and P 2 in the other view Drawing correspondences randomly from the two point sets is futile: Pr(correct)=min(1/ P1,1/ P2 ). We need to find a subset S P 1 P <latexit sha1_base64="938m1vwarb6cjie+a/ltfsy4dlm=">aaac93icjvhlssnafd2n7/qqunqtlikrklrbl6iblxwtfayuytjqyf5kjkip/q937sstp+bw/0d8a/0l74xrqii6icmzc+85m/feia2l0p73xhkghkdgx8ynypnt0zozlbn5a5xkgrdnnorjdhgwjuizi6awohshaszyfisifvxsm3jrumrkjvg+7qbiogjnstyvngmiopv6o2l6nlowt9dvqzxqqn8yjx7hb2szctvi1tuvqlfz7hj/ar8avrsrkvse0myjendkicaqqxmowadooyipdylxx+grlxgsni7qr5m0owujymdextd3jhzhbrvt3ngqq+z0skhvrkoxy6rjkc8jbe5zbty3zob9zbtnpc3duvqpcq+iwi1zyv/sfwt+v2dq0tjfhq1buk2pzux1vhdjbvfmzd2bqjq5pmqzfelxjdc3yo8+u1ajbo2mt8zgx2ymyc2ef7k5xs0tacd+93h+baf1mr9a83fxqptbxajhsyglrna817gjhttqjo8r3ombj07xuxzunnv3vkduabbwztl3b78ipte=</latexit> 2 of likely correspondences, or tentative correspondences Typically done using some heuristic!48
RANSAC for F Pick 8 random correspondences from S We do not know if they really correspond, but this can be tested: 1. Use the 8-point algorithm to estimate F 2. Check how well F matches each pair in S 3. Collect those that fit well into the consensus set C 4. If C is sufficiently large: F is OK: keep F and C Iterate r times!49
<latexit sha1_base64="fzkxcv47ivvzgcxl4u2+2myz6ho=">aaac5hicjvhlssnafd2nr1pfuzcudbzbf5zebd0irteupiktglvjxmkntzmwmsildononbj1b9zqt4h/oh/hntefh4hoshlm3hvozl3xiwm/kbb9kjmgboegr/kjhbhxickpc3qmlkspylzkoiasr56b8mapevx6mubhsebu2wv4odfaufhdsy4spwopzcfmj223gfonn7msqdnzxmzvg8jl3xoqnzeclxi514dxp3vlvtozajdsvayfwmlaedmqroyz6jhhbiyubxcekiqduejooyydgzfxj+gsjwj5os7rq4g0kwvxyncjbdg3sbvjja1przwtrwz0skcvikwfrdjelcciq9mshu+1s2j/8+5qt3w3dv29zktnrmqfsx/p+pn/1alajbry1dx4vfosgvudy1xs3rv1c+ttvzicyuiupqe4imy0st9ns2ssxbvqravjrzptswrpstwub+qwngdn+zh/gtpqyvkrofvrxfj2nuo85rcajzrnbsryrqvv8r7gax7xzdsmg+pwuptinxkzzhzflnh/do4imyk=</latexit> Probability of success Let w be the fraction of inliers in S In each iteration we pick N points that are all inliers with probability w N (approximately) The probability of not all N points are inliers is then given by 1 w N The probability of not all N points are inliers in r iterations is (1 w N ) r The probability that in iteration r, at least once, all N points are inliers: p = 1 (1 w N ) r Solve for r : log(1 p) r = log(1 w N )!50
Minimising geometric errors The 8-point algorithm uses an algebraic error. We would like to also estimate F using ML. This requires us to use an error that corresponds to a likelihood. We know that image coordinates from detection are well modelled as normally distributed. Via the negative log we can then obtain a leastsquares problem. Thus, we can look for a geometric error, but will all geometric errors lead to independent normally distributed likelihoods?!51
<latexit sha1_base64="nfpwislmirbu9vli/kuwgcz3iry=">aaadnhicjvhlssnafl2nrdb6qrp0eyycopskcroriok4eklgrddakkrthfikmxfkyg/5jykcoxf04w+48c4ytsqie5kcofecm3nnrncljgvabu4zyrdgx4rjpynjqemz8uzccqviyczno3cd6nqygxgpt5qccpechhexpcsllcvzffxwjykydfxjpgjjmwee+7rpbzmjzzsddgkzdrhq2x0we0bibotp91dtgunjif1oolzfhargojvpmpzspr9czulxurxv79lap7r7pbyayqnc0aqahoppogegatloboub6eaparahbg8i+maru2acw6cnomgqincgcxiriirrbfiootdgfugfiayd33octtpwx7niznjt4youvhe6vvhct4c6clfytzx1wcyl9rfsrgakvq3wb2vzhriclpd9y/eu/k9p9mkhd1uyb4o9hzir3dlzsixprexcheqky0kinma9reeibel8p2dvepjsxzytkeupuilymbczbqxpypd4wfr36/wjtmpvfb2qh21u6jvzvrdharzhge9ze+qwdw1oyvy1votyuyjypdwp98rdm1tjzz55+dku51f+8rmi</latexit> Geometric errors for F Given a set of N corresponding image points {y 1k, y 2k }, k = 1,..., N We can formulate a geometric error as NX 1 = d PL (y 1k, Fy 2k ) 2 + d PL (y 2k, F T y 1k ) 2 k=1 Epipolar lines! where dpl is the Euclidean distance between a point and a line Referred to as an L 2 error!52
<latexit sha1_base64="mwso2suxfq1jrb6erajkzuw3yqw=">aaadjhicjvhps9xafp6mbf3rx1s9egldsi0ts7iw6kuqvxisfvxxchddkp21q36smrswkh/h/6s33ktei2el13r3zticupr2qpl3vvd938ybf+qxf9jxlmas2wfpx8znlyy+fpx6zdvgu6udkzvfylphfmffyealfvoudswxmtvmc+ynqcx6qbst6r0frba8s/fljgedxd9o+zihvitia3h9lgsez+lgx5sjv0ubbj3ctude1enuq1u/gnut2qvcqp5iko9e1y7qt8p256ek9iosq0leo+m0hl3s6ca1qrnmdblgofoyiuoiegkyukiky/gq9bzbhyocsaeqwgqkuk4z1fgkbuksrgyf0ii+x5qdgtslxhkkrq5pl5jegpq2ppami15bsdrn1vvsoyv0b96v9lrnm9a/mf4jorlfcf2x7p75vzrvi8qy67ohtj3lglhdhcal1leitm4/6eqsq06yikdulygotfl+nm2tebp3dbe+rl9ppkjvhhpuiwt1shqw+3sc08fbu+wutdy9r83nltpqeazgpvzpnt+wir100cxvn/inp7ixtqxf1ql1dke1zoxmgy+wdxklf0i1ea==</latexit> Geometric errors for F This error does not take into account that the epipolar lines also, are perturbed by noise As an alternative, we want to find A fundamental matrix F A set of auxiliary points {y 1k, y 2k } where (y 1k ) T F y 2k = 0 (exactly!) for all k such that = NX d PP (y 1k, y2k) 0 2 + d PP (y 2k, y1k) 0 2 k=1 is minimised d PP is the Euclidean point-to-point distance!53
Re-parameterisation of F This means that we want to optimise over combinations of F and the auxiliary points that always satisfy the epipolar constraint! Cannot be done directly in a simple way Instead: Let F be determined from C 1 and C 2 Let the auxiliary points be the images of some virtual 3D points projected through C 1 and C 2 The auxiliary points and F are then always consistent Vary the 3D points and C 1 and C 2!54
Cameras from F Each camera matrix has 11 degrees of freedom: two cameras have 22 DOF F has 7 DOF There are many combinations of two cameras that produce the same F!55
<latexit sha1_base64="bw8apxifa8cxm0ratm5xknxqoiw=">aaac/hicjvhlssnafd3g97vq0k2wck5kuow6eyqcufroq9dwkqttojqvkokgnf6jo3fi1h9wq2vxd/qvvdomyc2ie5kcofeem3pvtuoxx8iw3sa08ynjqemz2bn5hcwl5dzkai0okshhvsdwg+jmtmlmcp9vbrcuowsjznm2y07t3ogmn16ykoabxxfxiwt6vtfnhe5ygqhwrtrv2b39ig0v9+p1hvna6hfntnlqco6xwhgh6xnlejiayxsfqy19fjgzycnbx0hufq20ecbbag8mpgrhfxzieuowysakrok+crehruimkezim1awowyl2b59u7srz6xpe+kzk7vdp7j0rqtusumagpiiwvi0xcut5szz37z7ylpe7yr+dublestwqexfukhmf3wyfoeodlunngokfsorczkxrhvf3lz/vpugh5a4idsujwg7sjnos640sapd9tzs8xevkvm5d7lcbb/yljrg8+c4r0gtwdc3c+bjtr68n416buvywbbns4qyjncmknnf4gnpenfutdvtxnv4stxgms0ahpb2+ambaavv</latexit> Cameras from F Choose C 1 = [ I I 0 ] (set WCS=CCS for camera 1) A convenient choice of C 2 is then given by C 2 =[[e 21 ] F T e 21 ] (why?) Note that e 21 is given by F e 21 = 0 C 1 = C 1 H and C 2 = C 2 H, for arbitrary 3D homography H, also gives the same F (why?)!56
Gold Standard estimation of F Determine an initial F 0 using a linear method From F 0 : determine initial C 1 and C 2 From {y 1k, y 2k } and C 1, C 2, triangulate virtual 3D points x k Relative to some arbitrary 3D coordinate system Projective triangulation Re-project the 3D points through C 1 and C 2 to the auxiliary points {y 1k, y 2k } These are consistent with F 0 Minimise ℇ over the 3D points x k and C 2 (C 1 is fixed) Use a non-linear optimisation tool, e.g., lsqnonlin ℇ is referred to as an L2 reprojection error The resulting C 2 (+ C 1 ) gives F optimal that Minimises a geometric error Maximises the likelihood of the correspondences under Gaussian noise This is referred to as Gold Standard estimation of F Computer Exercise 3!57
Summary The math introduced here and in the next two lectures builds on TSBB06 Multidimensional Signal Analysis The Maximum Likelihood(ML) trick is to look for models that make observations likely. Optimal Triangulation, and the Gold Standard method to find F are two examples of ML. More examples will follow The RANSAC algorithm is an important tool for handling data with outliers. Other tools are clustering and mode finding (LE6).!58