Doan Van Tuin vd Dig Tgp chi KHOA HQC & CONG NGHE 162(02)165-71
GIAI PHAP NANG CAO TOC DO CUA THUAT TOAN BELIEF PROPAGATION CHO H £ THONG THI GIAC N 6 I
Doan Van T u i n ' ' , BAI Trung ThSnh', H i HQu Huy^
'Trudng Dgi hgc Suphgm Ky thuat Hung Yen. 'Vien Khoa hgc vd Cong nghe Quan sic
TOM TAT
Thi giac noi (Stereo vision) la mot Uong chu d l quan trpng trong thj giac miy (Computer vision) Hien nay dg c6 nhieu tac gia nghien ciiu va thyc hien ky thuSt thi giac noi vdi nhilu giai phap khde nhau. Mgt trong cac giai phap thyc hien thj giac ndi c6 hi^u qua la diing trudng nglu nhien Markov MRFs (Markov Random Fields). MRFs la sy ket hgp giua ly thuyet xic suat va mo hinh d6 thj de tgo ra cac thuat toan suy dien gin diing nhu thugt toan GC (Graph cut) va thuSt toan BP (Belief propagation). Bai bao n^y chi ra m^t s6 mo hinh ciia MRFs va de xuit thu^t toan CFBP (Coarse to Fine Belief Propagation) d l thyc hiSn k j thuat thi gi^c noi vdi anh stereo camera co dp phan giai cao. Tac gia thuc hien mo phong va so sanh t6c dO cua thuat toan dl xuit CFBP vdi thugt toan BP tieu chuin. Kit qua chi ra ring thuat toan d l xuit n3ng cao dugc t6c dp xu ly gap 2 lan so vdi thuat toan BP ti8u chuin.
Ttr khoa: Trudng ngdu nhien Markov, Thi giac may, Thi giac ndi, MRFs, Thugt loan BP.
GIOI T H I E U C H U N G
Thi giac ndi (stereo vision) la ky thuat xac dmh khoang each tir camera den vgt, nhgn dang va tai tao lai vat d u a tren thdng sd sai l^ch cua di^m t u a n g ling {stereo corresponding) tren hai anh ciing chup mgt quang canh (stereo camera). Do vay tat ca cac nghien ciiu ve ky thuat thi giac ndi thyc chat la xac djnh ban d o sai l?ch (disparity map) cua anh stereo camera. K h o a n g mpt thap ki g i n day da cd rat n h i l u tac gia quan tam nghien ciiu. Hien nay cd rat nhieu giai phap d u g c d u a ra de t h u c hi?n ky thuat thj giac noi tit anh stereo camera dugc lay tir c o sd dii lieu c h u i n [1],[2]. Mgt trong cac giai phap la sii dyng M R F s n h u la mgt cdng cu de tgo ra cac thugt toan suy d i l n g i n dung.
M R F s la su ket hgp cua ly thuyet xac suat va md hinh d6 thi se cho ra mgt sd mo hinh Markov de giai quyet cac van de ve thi giac ndi [3]. Co hai dgng md hinh Markov chinh la
X I
,1,',1'
t
m o hinh dang ludi vdi ket cau mit cd 4 hen ket n h u Hinh l.a, mit cd 8 lien ket nhu Hinh l.b va mfl hinh dgng cay nhu Hinh I.e. Cac mit trong hinh duoc xem n h u la nhan bieu dien dg sai lech ciia hai diem t u a n g li'ng trong anh stereo camera. Trong md hinh dd thi xac suat thi cd hai logi la cac mo hinh do thj co hudng hay cdn ggi la mang Bayesian va mo hinh do thj v6 hudng hay cdn ggi la MRFs.
Mdt md hinh k i t hgp ca dd thj cd hudng va do thj vd hudng ggi la mo hinh do thj he sd n h u Hinh l.d. Cac thugt toan suy dien dugc hinh thanh tii M R F s la thugt toan SA [4], thugt toan ICM [5], thuat toan G C [6], thuat toan BP [7],[8] va thugt toan P Q B O [9]. Cd hai van de cua thi giac noi ma cac thugt toan suy dien can d u g c giai quyet la tlm x a c suat toi da hau nghiem ( M A P : Maximum a posteriori) [7] va x a c dmh tdi thieu hoa nang l u g n g chi phi ciia moi thong diep d u g c truyen [8].
(a) (b) (c) (d)
Hinh 1 Cdc md hinh MRFs vd dd thi he sd
Email- tuandv ute@gmail C<
Doan Van Tuan vd Dtg Tap chi KHOA HQC & CONG NGH$ 162(02): 65-71 MO HINHTRUCJNGNGAUNHEN MARKOV
Cho md hinh dd thj nhu Hinh l,G ^ (V. E) trong do G la mo hinh do thi bieu dien ban do chenh Ifch hay ban do do sau ciia cgp anh thi giac ndi, Vvk E tuong ling la cac niit va cac cgnh ciia do thi. Niit la nhan bieu dien sy sai lech cua cap diem tucmg ddng trong cap anh thi giac ndi, canh bieu dien nang lugng phai tra ciia mit vdi cac mit lan can cua nd. Ddi vdi moi mit / (i € V), xet X, la biln ngau nhien ciia nut i, X, la sy chuan boa ciia X, va X, la khdng gian trgng thai cua x, (x^ X). Xet X = (X)^v la bien ngau nhien lien ket va x= (x^^y la su chuan hoa cac gia trj md hinh do thi trong khdng gian X.
MRFs la sy ket hgp giiia ly thuyet xac suat va ly thuyet do thi, no la md hinh do thi G vd hudng nhu Hinh 1 .a, Hinh 1 .b va Hinh 1 c, tat ca cgnh E la vd hudng va mit dgc lap vdi tit ca cac mit lin can ciia no:
\li^V,X,LXy_^^\X^^ (1) trong dd N, = {j\{ij}^} la t^p cac mit lan cgn
ciia mit i trong dd thj G. Mgt khai niem quan trgng trong MRFs la triim (clique), nd dugc dinh nghia la tat ca cac ket noi trong mgt tgp con cua cac mit trong dd thi. Theo ly thuylt Hammersley - Clifford [10], xac suit cac he sd duoc xac dmh la:
P^^) = ^Y\fMc) (2) trong do Z la he so chuan hda, i/z^ix^) la ham
kha nang ciia triim c, C la tap ciia cac triim trong dd thj G.
Nang lugng cac he so
E(x) = Y.eXx,) (3)
trong do 0^(x^) =-\ogi//^(x^) . Viec xac dinh t6i da xac suit hgu nghi?m MAP tuong duong vdi xac djnh toi thieu ham nang lugng chi phi [3].
Hinh l.a va Hinh l.b bilu diln md hinh MRFs dang ludi vdi hai cau tnic la mit cd 4
ket ndi va mit cd 8 ket noi. Cac mit x la cac nhan bilu dien su sai lech ciia cgp dilm tuong ting ctia anh stereo camera. Da co rat nhilu thuat toan sir dung mo hinh dang ludi dl giai quyet cac van de ve thj giac [7],[8],[11],[I2].
Xac xuat hgu nghiem MRFs dugc xac djnh la;
p(x) = lly/,(x,)YlYlwiM,) (4)
x ' = a r g m a x p ( x ) (5) trong dd il/,(^,) 's xac suat ciia nut i neu d$
tin cay mit / la cao nhat thi xac suat i^/Xx,) la cao nhit, i//^^ (x ) la xac suat cua mit / vdi niit J lan can mil /.
Nang lugng MRFs dugc xac djnh la:
£{x) = X e , ( x , ) + i ; 0,,,(x,) (6) X = a r g m i n £ ( x ) (7) trong dd 0,(Xi) '^ "^"g lugng chi phi cho nut / va dugc ggi la ham gia tri chi phi dir li?ii {cost data). ^,.(X|j) la nang lugng chi phi cho nut / vdi niity lan cgn vdi mit / va dugc gpi la ham gia tri chi phi dp nhan {cost smooth).
Ham gia trj chi phi dg nhan dugc xac dinh:
e^^{x,^) = p{x-x^) (8) p{x,-x^) = min(c.]x^ - x j , . : / ) (9)
trong do c la he sd tang thudng chpn c = 1, d la dinh dii'ng do tang trong mo hinh tuyen tinh.
Hinh 2 bieu dien md hinh MRFs an, trong do nut X, la cac nut an va mit z, la cac nut quan sat [7],
Z, Z2
Hinh 2. Md hinh Markov dn Xac xuat hgu nghiem MRFs dugc xac djnh la:
pM=nf^:(^..''.)nnn(''.'''.) c"'
Doan Van TuIn vd Dtg Tgp chf KHOA HOC & CONG NGHE 162(02): 65-71 x - a r g m a x o ( x ) (11)
xeX
trong do ^i/,(2,,x,) la xac suat ciia mit quan sit z, den mit in x„ ^i/y(x,,x^) la xac suit ciia nut an x, vdi nut an Xj lan can mit x,.
Nang lugng MRFs dugc xac dinh la:
£Cx)=x;^,(z„x,)-t- 2 ^,/x„x^) (12)
X* = a r g m i n £ ( x ) (13) trong do ^^(z,,x,) la nang lugng chi phi cho
nut quan sat z, den mit in x, va 5,j(x,,x^) l i nang lugng chi phi cho mit Xi vdi mit Xj lan can vdi x,.
Hinh 1 .c bieu dien md hinh MRFs dgng cay la dang do thi vd hudng. Thdng diep dirge truyen giira cac niit, cac mit la cac nhEn bieu dien cho sy sai lech ciia cap diem tuang dong, Thuat toan suy dien nhu trong cac nghien ciiu [13,14,15], thyc hien chuyen thong di?p giii'a cac mit cua cay. Thdng diep m,^^ tii mit i tdi mit j dugc tinh toan nhu sau:
»^,^j(Xj) = maxP{Xj,x^) Y[ ^k^,{x,) (14)
trong dd N^{r) la mpt tap ciia tat ca nut i.
Nhan MAP ciia bien tgi mit gdc r la:
x ' = a r g m a x Y\ ^k-*r(^r) (^^J Cho nhan MAP x'ciia biln X^, nhan MAP cua nut i dugc xac dinh nhu:
x'=msKp{x^,x,) n «*-.*(*,) ^'^*
THUAT TOAN SUY DIEN LAN TRUYEN DO TIN CAY BELIEF PROPAGATION Xet mo hinh MRFs an nhu trong Hinh 2, trong do Xi la cac niit in, Z, la cdc mit quan sat, His, msj la cac thdng diep truyln.
Xac suit hau nghiem dugc xac dinh la:
P{XlZ) = Y[y/,{^„z,)UU^'Mr^J^ <'^'
trong dd ^y^(x,,z,)la xac suit ciia chinh ban than nut x, hay cdn gpi la gia tri cd kha nang xay ra nhit, 1^ (x^jX ) la xac suat giu'a mit X, vdi niit lan can Xj cua nd. yJ,.{Xi,Xj) la gia tri trudc va dugc gpi la ham ben viing.
Neu dp sai l?ch d, co cac gia tri sai lech la L thi ^i/,(x,,Z,) la vec to co dp dai la L va
^ (Xj,X ) la ma tran tuang iing vec to LxL.
^,(Xj,Zj) va \(/ {x^,x) dugc xac dmh [7].
Dat m,(x,.z) = m,(x) = ^,(x^,Z,)la thdng diep glii tu z, den x, Dgt m,j(x„Xj) = m,j(xj =
y/y{x^,Xj) la thdng diep giri tii nut ;i:, den mit lan can x,, va do tin cay nhan x, la b,(xj. Tat ca m,(xj, m,j(xj) va b,(xj la cac vec ta co dp dai la L,
Thuat toan BP theo chuan tich eye dai {max- product) nhu sau:
Bu&c 1: Dat gia tri ban dau ciia thdng diep my(xj) va m,(xj.
Budc 2' Tinh toan thdng diep cap nhat m,j(Xj) vdi vdng lap t = 1 :T
m'*'(x )«-frmaxi^,j(x,,x )m,'(x,) ] T wj,(x,) (18) Budc 3: Tinh toan dp tin cay
b^{x_)-<^kmXx,) n ^MX^,) <'9>
gmaxb^x,) (20)
Xem xet Hinh 2, thong diep mdi gii'i tfr niit X|
den X2 va dugc cap nhgt nhu:
m"^^ <- kmax^ W\2(X[>^2)"^]'"i.i'^4.i'^5.\ (21) Dg tin cay dugc tinh nhu:
k la he s6 chuan hda
DE XUAT THUAT TOAN CFBP VA KET QUA MO PHONG
67
Doan Van Tuan vo Dig Tjp chi KHOA HOC & CONG NGHE 162(02): 65-71'
, ^
Md ta thu^t toan
Thugt toan BP hogt dgng dya tren nguyen ly lan truyen cac thong di?p ciia cac niit thong qua cac vong lap, mdi vdng lap tung ting vdi mdt miic do sai lech ciia diem tuong ting trong anh stereo camera. Thugt toan BP c6 uu dilm la do tin cgy cao tuy nhien miic dp phiic tgp tinh toan Idn va yeu cau bd nhd cao. De thuc hien thugt toan BP nhanh can giai phap la xii ly song song tiic la mdi vdng lap dugc thyc hien d6ng thdi tren tit ca cac mit tuong li'ng vdi mdt mii'c do sai l$ch ciia diem anh.
Ngoai vin dl xir ly song song cdn phai giam sd lugng mit khi dd giam dugc do phiic tgp ciia tinh toan ciing nhu yeu cau cua bd nhd.
Dya tren dgc dilm do sai lech ciia hai diem tuong ling ciia anh stereo camera ta thay ddi vdi cac diem anh cang gan camera thi do sal
lech cang Idn va ddi vdi cac diem anh cang xa camera thi do sai lech cang nhd.
Tit nhii'ng phan tich tren chiing toi de xuit thugt toan CFBP {Coarse to fine Belief propagation) vdi nguyen ly lam viec nhu sau:
tai mdi vdng Igp khi thuc hien viec lan truyln dg tin cay nhu Hinh 3, xen ke giifa hai vong Igp se thuc hien giam sd lugng diem anh theo phuong phap C^{Coarse to fine) miic 2 nhu Hinh 4. Mdi vong lap tuang ling vdi mgt iniic dg sai lech cua hai diem tuong ling trong anh stereo camera se dugc thyc hien lan truyen dp tin cay dong thdi giiia cac nut nhd su dung xii ly song song tren cau triic cua GPU diing phan mem CUDA. Nhu vgy, cii sau moi vong lap thi sd lugng diem anh se giam di 4 lin CO nghTa la so lugng phep tinh va y^u cau ve bg nhd se giam va toi thieu ve nang lugng chi phi.
i^^KD
t i t !
t t t f H H
• • • •
X -
• . •! • •
Hinh 3 (a) Cdu true thong diep cua BP, (b) Sadd Hinh 4. Phuang phap tic tho den tinh muc thdng diip lan truyin cho mot vdng lap 2(Coarse to Fine) X = \_X[, X\, X',, X\ ] Cac budc thuc hien thuat toan
Buac 1: Vdng lap thii nhat thyc hien ddng thai truyen thong diep giira cac niit theo Hinh 3 vdi diem xuat phat ban dau la 0,0.
Nang lugng chi phi dugc xac djnh theo cdng thiic sau:
Thong diep cap nhat tai vong lap t dir(?c xac dinh la:
(23) (24)
trong do E(i)\j la tap nut ISn c^n i ngoai trirj. Sau T vong lap thi dp tin cay cua moi niit la:
Nut x^ dugc lira chgn va xac dinh theo cong thuc:
x' = argmin6^(x^) 68
(25)
Do^n van Tu2n va Dtg Tgp chi KHOA HQC & CONG NGHfi 162(02): 65-71 Ham nang lugng chi phi dp nhin dugc xac
dinh theo md hinh tuyIn tinh.
V{x, -Xj) = min(c|x, ~Xj\,d) (26) trong do c la ti 1^ tang ciia ham nhin, d la ngu'Sng diing tang cua ham nhin Khi dd thdng diep cgp nhat dugc xac dinh la:
= m^in(min(c|x,-Xj|,(/)+^,(x,)+ J ] m!ll,(^,)) (27) Buac 2: Sau khi thyc hien xong budc 1 se thuc hien viec giam sd lugng diem anh theo phuang phap CT miic 2 nhu Hinh 4 dung giai phap xii 1^ song song.
Nang lugng chi phi CT miic 2 dugc xac dinh la:
E{X)=Y^e,{x:)
(28) Sau khi thyc hien xong budc 2 se thuc hien budc 3 vdi vdng lap thii' 2, tuy nhien so lugng diem anh giam di 4 lan so vdi vong lap I.Tiep tyc thuc hien cac budc le theo budc 1 va cac budc chan theo budc 2 cho den het dai mii'c dp sai lech ciia cap diem anh tuong ung trong anh stereo camera. Vi du vdi camera Bumblebee XB3 [16] cd dp phan giai cao 1280 X 960 va 400 miic dg sai lech vdi toe dp 16 FPS thi theo thugt toan BP tieu chuan vdi cau triic dgng ludi cd 4 lien kit cho mdi mit thi yeu cau bg nhd la 4.6GB RAM con thuat toan CFBP yeu ciu gin 40MB RAM, Danh gia ket qua mo phong
(c) (d) Hinh 5. Kit qua thi/CC hiin bdn do chenh lech
Kit qua thyc hien ban dd sai l?ch nhu trong Hinh 5, Vdi thdng so ciu hinh CPU core i3, 8GB RAM, GPU Nvidia GTX 750 Ti cd 460 loi va 2GB RAM. Anh stereo camera cd do phan giai cao 1280 X 1110 thugc tap dii lieu [1]. Hinh 5.ava Hinh 5.b tuang ling la anh trai va anh phai ciia anh stereo camera. Hinh 5,c la ban dd sal l?ch theo thuat toan BP tieu chuin, Hlnh 5.d la ban do chenh lech theo thugt toan CFBP. Tii Hinh 5.c va Hinh 5,d, chiing ta nhan thay toe dp thuc hien ban dd sai lech theo thuat toan CFBP la 2 ban dd sai l?ch trong I giay, thugt toan BP tieu chuin la I ban do sai lech trong 1 giay (l6c do th^ec hi?n dupe hien thi goc tren ciing hen Irai cita ban do sai lech) nhu vay CFBP nhanh hon BP.
Doan Van Tuan va Dtg Tgp chi KHOA HOC & CONG NUMb iozi^u.ij:o3- n
K E T LUAN
Thi giac ndi la k j thugt xac djnh khoang each tir camera den vgt va tai tao lgi vat dya tren thong tin ve sy sai lech ciia diem anh tuong ting trong anh stereo camera. Da cd nhieu thugt toan dugc de xuat de xac dinh ban dd sai lech. Mpt trong cac thugt toan la thugt toan suy dien g i n diing BP dya tren mo hinh trudng ngau nhien Markov MRFs. Trong bai bao nay, chiing tdi tgp trung phan tich cac md hinh M R F s va d l x u i t thugt toan BP cai tiin la CFBP. Qua k i t qua md phdng cho t h i y thugt toan CFBP cd tdc do thyc hien ban dd sai lech g i p 2 lan so vdi thuat toan BP tieu chuin ngoai ra thugt toan CFBP da giam dugc y£u cau tinh toan ciing nhu yeu cau bp nhd so vdi thuat toan BP tieu chuin. Tuy nhien viec Igp trinh thuc hi?n thuat toan CFBP khd khan va dd chinh xac cung giam so vdi thuat loan BP tieu chuin. H u d n g nghien ciiu tiep theo la viia nang cao d u g c tdc dg thyc hien ban do sai l|ch ma dg chinh xac khong bi suy giam.
TAI LIEU T H A M K H A O 1. D Scharstein and R. Szeliski, httpV/vision middleburv.edu/stereo/eval/.
2. Stefano Mattoccia, (2012) "Stereo vision:
Algorithms and Applications," Lecture Note, University of Bologna,
3. R. Szeliski, (2010). "Computer Vision: Algorithms and Applications," Springer-Verlag New Yoik hic.
4. S. Geman, D. Geman, (1984) ''Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images," IEEE Transactions on Pattern Analysis and Machine Intelligence 6 (6) 721-741.
5. J. Besag, (1986) "On the Statistical Analysis of Dirty Pictures" Journal of the Royal Statistical Society (Series B) 48 (3) 259-302.
6. Y. Boykov, O. Veksler, R. Zabih, (2001) 'Fast Approximate Energy Minimization viaGraph Cuts," IEEE Transactions on Pattern Analysis and Machine Intelligence2i (1IJ 1222-1239.
7. J, Sun, N, N, Zheng and H. Y. Shum, (2003)
"Stereo Matching Using Belief Propagation,"
IEEE Trans. PattemAnalysis and Machine Intelligence, vol. 7, no. 2 5 , , pp, 787-800.
8. P, F. Felzenszwalb and D. P, Huttenlocher, (2004)
"Efficient Belief Propagation for Early Vision," Proc, IEEE Int.Conf. Computer Vision and Pattern Recognition, vol. 1, no. 70,, pp. 261-268.
9. C. Rother, V. Kolmogorov, V. Lempitsky, M.
Szummer, (2007) "Optimizing Binary MRFs via Extended Roof Duality," in: IEEE Conference on Computer Visionand Pattern Recognition (CVPR).
10. J. Besag, (1974) "Spatial Interaction and the Statistical Analysis of Lattice Systems," Journal of the Royal Statistical Society. 36 (2) 192-236.
11. V. Kolmogorov, R. Zabih, (2002) "Multi- camera Scene Reconstruction via Graph Culs,"\t\:
European Conference on Computer Vision (ECCV).
12. S. Roy, I. J. Cox, (1998) "A Maximum-Flow Formulation of the N-camera Stereo Correspondence Problem," in: IEEE International Conference on Computer Vision (ICCV).
13. A. Blake, P. Kohli, C. Rother, (2011) "Markov Random Fields for Vision and Image Processing,"
MIT Press.
14. M. J. Wainwright, T. S, Jaakkoia, A. S Willsky, (2005) "MAP Estimation via Agreementon Trees: Message-passing and Linear Programming," IEEE Transactionson Information Theory 51 (11) 3697-3717.
15. V, Kolmogorov, (2006) "Convergent Tree- reweighled Message Passing for Energy Minimization," IEEE Transactions on Pattern Analysis and Machine Intelligence28(10) 1568-1583.
16. P. Grey. Bumblebee xb3 camera.
http://www,ptgrey,com/products/bbxb3/indexasp.l
Doan Van TuIn vd Dtg Tap chi KHOA HOC & CONG NGHE 162(02): 65 - 71
S U M M A R Y
A D V A N C E D S O L U T I O N S S P E E D O F B I L I E F P R O P A G A T I O N F O R S T E R E O V I S I O N S Y S T E M S
Doan Van T u a n ' ' , Bui Trung Thanh', Ha Huu Huy^
'Hung Yen University ofTechnology and Education, 'institute of Military Science and Technology Stereo vision has attracted many attention in recent years. There are many authors doing researches and implementing stereo vision systems with diffirent methods. One of the most useful method for stereo vision is using Markov Random Fields (MRF). MRFs is combination of probability theory and graph model to create inference algorithms. These algorithms are implemented on embedded systems. In this paper, we present some MRFs models and propose a coarse to fine belief propagation (CFBP) algorithm for stereo vision with dense image pairs. We simulate and compare speed of conventional BP and CFBP. Simulation serults show that the proposed algorithm has better speed in compared with conventional BP
Keywords: Markov random field. Computer vision, Stereo vision, MRFs, BP algorithm
Email, tuandv.utei
