Tap chi Khoa bgc Tnrang Dgi hgc Cdn Tho PhdnA: Khoahgc Tunhien. Cong ngbe vd .Moi inrdng: 29(2013/- 1-7
Tap chf Khoa hpc Tru'dng Dai hpc Can Thd website: sj.ctu.edu.vn
PHAN L 6 P A N H VOII GIAI THUAT GIAM GRADIENT NGAU NHIEN DA LOfP
D6 Thanh Nghi' va Pham Nguyen Khang'
' Kltoa Cong nghe Thdng tin & Truyin thong, Trudng Dai hgc Cdn Tha
Thdng tin chung:
Ngdy nhdn: 17/04/2013 Ngdy chdp nhdn: 24/12/2013
Title:
Classifying images •with multiclass stochastic ^adient descent algorithm
Tir khda:
Biiu diin dgc tnmg khong ddi SIFT, Md hinh tiii tu Bo VW, Mdy hgc vec ta hd trg SVM, Phuang phdp gidm gradient ngdu nhiin SGD
Keywords:
Scale-Invariant Feature Transform - SIFT, Bag-qf- Words - BoW, Support Vector Machines - SVM, Stochastic Gradient Descent - SGD
ABSTRACT
In this paper, -we present a new algorithm, MC-SGD (Multiclass Stochastic Gradient Descent), to effectively classify multiclass images. The representation of the images is based on the bag-of-words (BoW), which is constructed from the local descriptors (the Scale-Invariant Feature Transform method - SIFT). The pre-processing step brings out datasets with a very large number of dimensions. We propose a new algorithm called MC-SGD that is suited for classifying very-high-dimensional datasets. The numerical test results on a real dataset showed that our algorithm MC-SGD outperforms Support Vector Machines (SVM) using non-linear kernel functions (Radial Basis Function - RBF).
TOM TAT
Trong bdi ndy, chung tdi trinh bdy gidi thudt mdi, gidm gradient ngdu nhien (Multiclass Stochastic Gradient Descent - MC-SGD), cho phdn ldp hiiu qud dir liiu dnh da lap. Tdp dir liiu dnh bieu dien dnh bdng md hinh tdi tir (Bag-of-Words - BoW) sir dung cdc net dgc tnmg khdng ddi vol nhirng bien ddi ti li (Scale-Invariant Feature Transform - SIFT), dua tren ddc trung cue bd, khdng bi thay doi trudc nhirng biin ddi li le dnh, tinh tien. phep quay, khong bi thay ddi mdt phdn ddi vdi phep bien ddi hinh hoc affine (thay ddi gdc nhin) vd mgnh vdi nhirng thay ddi vi do sdng, su nhiiu vd che khudt. Chung tdi di nghi mdt gidi thugt phdn ldp da ldp mdi, gidm gradient ngdu nhiin MC-SGD, eho phip phdn lap hiiu qud dii- lieu co sd chiiu ldn thu dugc tir budc biiu diin dnh. Kit qud thuc nghiem tren tap dir liiu thuc cho thdy gidi thugt MC-SGD phdn ldp nhanh, chinh xdc han khi so sdnh vdi gidi thugt mdy hgc vie ta hd tra (Support Vector Machines - SVM) sir dung hdm nhdn phi tuyin (Radial Basis Function - RBF).
1 GIOI THIEU
Phin lop inh la gin nhan ty dfing cho timg inh theo chu dl di dugc dinh nghia trudc dya vao ndi dung eiia anh. Phan lop anh co nhilu img dung frong thyc tl nhu nhdn dgng chir sd frln chi phieu ngin hang, ma sd tren bl thu ciia dich vy buu chinh, hay eac chir sfi fren cac bieu mau ndi chung, die bi|t la tfi chuc npi dung frang web mpt cich ty dfing bing each ddnh nhan ty ddng dnfa.
He thdng phin lop inh thudng bao gdm hai budc: nit trich dac tnmg tir ndi dung dnh vd huan luyln mfi hinfa miy hgc dl gan nhin ty ddng tir cdc dgc trung nay. Hieu qua cua h | thdng phan ldp phy thufic vio cic phuong phdp sir dung d hai giai dogn frln.
Cac nghiin ciiu trudc day (LeCun et al., 98), (Viola & Jones, 01). (Zheng & Daoudi. 04) sir dung tiep can nit trich dac tnmg dya tren phat hien ciia cac diira, mau sac. kit cdu (texture), td chirc
Tap chi Khoa hgc Tnr&ng Dgi bgc Cdn Tho Phdn A Khoa hoc Tu nhien. Cdng nghe vd Mdi tnr&ng. 29 i20l3): 1-7 dd (histogram). Mang no-ron (neural net^vorks),
mdy hpc vec to hd frg (support vector machines), gidi thuat boosting dugc huan luyen dl phan Idp inh.
Gdn day. mdt huong tiep can cua (Bosch et al., 06) dya vio phuang phap bieu diln inh bang cac net dgc trung khdng dfii vdi nhimg biln dfii ti le SIFT (Lowe. 04) vd mfi hinfa tdi tir BoW. Die tnmg cyc bd SIFT kfafing hi thay ddi frudc nhung bien dfii fr" II inh. tinh tien, phep quay, khfing bj thay dfii mfit phin dfii vdi phep biln dfii hinh hpc affine (thay dfii gfic nfain) va raanfa vdi nhihig thay dfii vl dp sang, sy nhilu vd cfae khudt. Mdt dnh dugc bilu diln bdi tip hpp tiii tii duge xdy dyng bang each ip dyng mdt giai thuat gom nhdm len cac vec to md id cue bfi SIFT. Giai dogn tiln xu Iy cfao ra mpt tap diJ lieu vdi sfi efaiiu rat lon. Tic gii dl xuat giam chilu dO" lieu vd sir dung k lang gieng dl phan ldp failu qud inh.
Chung tdi de xuit sir dyng y tudng ciia phuang phdp bilu diln dnh bang ddc frung khdng ddi SIFT vi mfi hinh ttii tir. Tuy nhien thay vl giam chilu thi chiing tfii de xudt mpt gidi thuat hpc mdi. gidm gradient ngau nhien MC-SGD, cho phep phin ldp hieu qud diJ lieu cd sfi chieu lon tfau dugc tir budc bilu diln inh. Kit qud thyc nghiem tren tap dfl- lieu inh thyc tir ImageNet (Deng et al.. 12) chi rdng giai thugt mdi MC-SGD phin ldp nhanh, chinh xic khi so sinh vdi giai thudt may hpc vec to hd frg SVM (Vapnik- 95) sir dung hdm nhdn phi tuyin (Radial Basis Function - RBF).
Phin tiep theo cua bai vilt dugc trinh biy nhu sau: phin 2 trinh bay ngdn gpn ve bieu diln anh bang md hinh tiii tir ciia die trung cyc bd khdng dfii, phin 3 trinh biy giii thuat pfadn lop da ldp MC-SGD. Phan 4 trinh biy cac kit qui thyc nghiem tiep theo sau dfi la kit luin va hudng phat trien.
2 BIEU DIEN DAC T R U N G KHONG DOI VA MO HINH TUI TlT
Bieu diln infa la mfit budc quan trpng frong phan loai anil. Budc nay cd dnh hudng rdt Idn din kit qua phan logi cufii ciing. Trong lanh vyc phan ldp va tim kilm anh, dac frung cyc bd SIFT (Lowe.
04) la nhimg diim dac trung, viing ddc tnmg di bilu dien anh rdt failu qua, ngay cing trd nin pfad bien. Nghien cuu tien phong cua (Bosch et al., 06) dc xuat he thing phan lop anh dya trin die trung SIFT vd mfi hinh tiii tir (xuat phdt tir >' tudng phin Idp van bdn). Giai doan bieu diln anh theo mo hinh tlii tir va die trung khfing ddi SIFT bao gdm 3 budc chi'nh: (i) phdt hiln va bieu dien cac net dac tnmg
cue bp, (ii) xay dung tu diln cdc lir tryc quan vd (iii) bilu diln anh dudi dang vec to tin sfi xudt hiln cdc lit fryc quan trong inh.
Hinh 1: Cde diem dac trung dirge phdt hifn bdi thudt todn Hessian-AfGne 6 budc ddu liln, dnh dugc dua vl dgng muc .\im. Cac diim ddc trung (Hinh 1) dugc tinh tren nhung inh niy bing cdch su dung cdc giii thuat phil faien diem dac tnmg cue bfi (local feature detector) nhu Id Harris-AfTme, Hessian-Affine (Mikolajczyk & Sclimid. 04). Nfaiing diim die tnmg nay cd the Id cyc frj cue bp ciia phep toan DoG (Difference of Gaussian) hoac la cyc dai ciia phep todn LoG (Laplace of Gaussian). Sau do, vimg xung quanh cdc diim dac trung dugc xac dinh vd mfi la bdng cdc vec to mo la cyc bp, Vec to mfi Id SIFT dugc ddnh gid rat cao bdi gioi chuyen mfin frong vi|c bieu dien cdc viing ximg quanh diem dac trung bdi vi nd khdng dfii ddi vdi nhiing biln dfii ti 11, tmh tien, phep quay, vd khfing dfii mdt pfain vdi doi voi nfaihig thay ddi vi gdc nhin, dfing thdi nd cung rit manh vdi nhiing thay ddi vl dp sang, sy che khuit, nhieu.
Hinh 2 minh hog mot vi dy ciia vec to md ta SIFT dugc xay dyng tir viing cyc bfi xung quanh mfit diim dgc trung. Mdi vec to md ti Id mfit ma frdn 4x4 cac td chuc do. Mfii tfi cfaiic dd cd 8 khoang tuong iing vdi 8 hudng. Do do. m6i vec to mo td SIFT la mdt vec to 4x4x8=128 chilu. Liic niy, mfii anh dugc bieu diln bdi mot tap cdc vec to mfi ta SIFT.
Budc ki tiip li thiet lip cic tir tryc quan tir cac md ta cue bfi da dugc tinh d budc trudc. Thudt giai i-means (MacQueen, 67) dugc thyc hien tren cac vec to md la di phan cdc vec to SIFT thdnh vao cac nhdm (cluster) va mfii cluster tuong iing vdi mdt tir tryc quan. Tip cde cluster nay tao thdnh mdt tir dien. Sau ciing, mfii vec to mfi id frong dnh sl dugc gan vao cluster gin nfait (dya vao khoang each mfii vec to din cdc tam ciia cic cluster dai dien da dugc dinh nghTa truoc dd). Tiip theo. mdt dnh sl dugc bieu dien bing tin sfi ciia cic tir fryc quan trong anh. Hinh 3 md id cac budc tgo md hinh BoW bieu dien cac anh.
Tap chi Klioa hgc Truang Dgi hgc Can Tha PhdnA. Khoa boc Tunhien. Cong nghi vd Moi truang- 29(2013) 1-7
^ ^ ^ ^
Hinh 2: Dgc trung cue bg SIFT dugc tinh todn tir>iing xung quanh diem ddc biet (vong tron):
gradient ciia anh (trdi), vec-to mo ta (pfaai)
Initial mages Interested points detection
— • ,—»• Histogram 1
— • o j — t . Htst(^ram 2
— > ^ —¥• Histogram n
Visual words 1
Clustering
Visual v/ords 2
Visual words k
Hinh 3: Tao mo hinh BoVW de bieu diln anh 3 THUi^T GLAI GIAM GRADIENT NGAU
NHIEN (MC-SGD)
Giai dogn tiln xii Iy cfao mfit tip dir lilu vdi sd chilu rit lon (vi du, 50000 tir tryc quan vdi nhieu die trung dau vao vdi moi dac trung chi chiia it thfing tin cho phin ldp). Gidi thudt mdy hoc vec to
hfi frg SVM (Vapnik. 95) la mfi hinh hieu qua va phfi biln cho van dl phan ldp nhung tap dii lieu cd sd chieu Idn. Xudt phdt tu cai dat hieu qud giai thudt SVM bdng phuong phap giim gradient ngdu nhien SGD (Bottou & Boussquet. 08), chiing tdi phdt trien giai thuat MC-SGD cho phdn Idp da ldp tap dir lieu cd sd chieu lon nay.
Tgp cbi Kboa bgc Tnr&ng Dgi hgc Can Tha Ph&n A: Khoa hoc Tunhien. Congnghe vd Moi tnrang' 29(2013): 1-7 3.1 Giii thuat mdy hoc vec to ho trg SVM
Xet vi dy phdn Idp nhi phdn tuyln tinh nhu Hinh 4. Cfao m phdn tu Xi, X2, ..., Xm trong kfadng gian n chieu, cd nhin (Idp) cua cac phin tir li.iv,
>•:, ..., .Vm cd gia tri I hoac -1. yi = 1. niu xi thudc Ififp +7 (Idp duong, lop chiing ta quan tdm), ,v/ = -1, nlu Xi thudc Idp -1 (lop dm hay cic ldp cdn lai).
SVM tim sieu phang tdi uu (xic dinh bdi vec ta phip tuyln vi' va do lech cOa silu phdng b) dya frln 2 silu phing ho trg cda 2 lop. Cac phan Hi lop +1 nim bin phii cua silu phang hd tra cho ldp +1. cic phin tir lop -I ndm phia ben frai ciia sieu phing fad frg cho Idp -1, Nhirng phan tii nim ngupc phia vdi silu phang hfi trg dugc coi nhu Idi. Khoang each loi dugc bilu dien bdi & ^ 0 (vdi xi nam diing phia ciia silu phdng hd trg ciia nd thi khoing each Idi tuang ling zi = 0, cdn ngugc lgi thi Zi > 0 la khoang each tir dllm xi den silu phing hd trg tuang iing cua nd). Khodng cich giiia 2 silu phang hd frg dugc gpi la le. Sieu phdng tfii uu (nam giiia 2 silu phing h§ trg) tira dugc tir 2 tiiu chi la cyc dai hoa II (Ie cang ldn, mfi fainfa phdn Idp cdng an loan) va cue tiiu hda Ifii. Vin dl din din viec giii bii loan quy hoach toan phuang (1):
min nw. b, z) - (1/2) ' » - - c^z,
(1) y,(w.x, -b) -\- z, >1 z, >0 (1=1,2 m) hdng c> 0 sir dung de chinh dg rong li vd ldi Giai bai toin quy hoach toin phuang (I), thu du'pc (w, b). Phan ldp phan tir x dua vao ddu ciia (wv-b)
A x^ w - h = -1 ,
<
lc = 2'||uj|
Hinh 4: Phan lop tuyen tinfa voi may hoc vecto ho trg
Bii toin quy hogch todn phuang (!) dugc nghien ciru phd biln frong loan tdi uu. Hiln cd nhiiu giii thuit tiiu bieu nhu LibSVM (Chang &
Lin, 01), SMO (Piatt. 98), Newton (Mangasarian, 01) diu cd dp phtic tgp bgc 2 vdi sfi phin tir dii lilu.
3.2 Gidi thugt giam gradient ngUu nhien (SGD)
Mpt cii dat cho gidi thuat SVM cua (Bottou &
Boussquet, 08) dya fren phuang phdp giam gradient ngau nhiin. cd dp phiic tap tuyln tinh vdi sd phan tii dii lieu. Bing cich thay thi r, bci w, x,.
y, (khdng xet dp lech b) til cac rang bupc vio ham muc tiiu cOa (1). viec tim silu phing tfii tru ciia SVM cd tfae dupe tfayc hiln bdi (2):
min 'F(w, x. y) QJl) \\w\\^ +(l/m)
^max{0,l-_yX"'-^,)} (2)
r = I
Phuong phip giam gradient (GD) thyc hiln tfii uu vdn dl (2) bang cich cgp nhat w tai ldn lip thii (t+1) vol tfic dp hpc 77,, nhu frong (3):
'(n,/m)Yy.¥i.'^n^,^y,) (3)
Phuang phap gidm gradient ngau nhiin (SGD) thuc hiln don giin budc cap nhdt w,+i chi sir dyng mpt phdn tir ngau nhien (x,. yl) tai mfii Idn lap:
w,,} =w,- TjiV»iff(w,, x,, yd (4) Cd thi thiy rdng gidi thuat SGD don gian, thyc
hiln cac budc lap, mdi budc lap chi liy 1 phdn tii ngdu nhiin tir tgp dir lieu, thyc hiln cap nhdt w thay vi phdi giii bii toan quy hoach todn phuang (1). Giai thuat SGD cd dfi phiic tap tuyen tinh vdi sfi phin tir cda lap dii lieu hpc, pfaan ldp dii lieu cd so phan hi vd sd chilu Idn rit faieu qua (Bottou &
Bousquet, 08).
3.3 Giai tfauat giam gradient ng^u nfailn cho phan ldp da Idp (MC-SGD)
SGD cung nhu hau hit cdc gidi thuit SVM diu xuat phat tir van dl phan lop nhi phin (2 ldp:
duang vi am). Chiing tfii md rpng giii thuat SGD de cd the giai quylt vin dl phdn ldp tap dii lieu co c lop (c > 3) hay con gpi la da ldp.
Top cbi Kboa bgc Tnr&ng Dgi boc Cdn Tlia Phdn -•) Khoa hoc Tu nhien. Cong nghi vd Moi tnr&ng 29 (2013) 1-7
Hinh 5: Pfain Idp da lop, 1-vs-alI (trdi), 1-vs-l (phai) Dl gidi quylt phdn Idp tap dii lieu da ldp,
gidi tfauat SVM thuong dya fren 2 phuang p h ^ don gidn Ii I-vs-all (Vapnik, 95) vd 1-vs-l (Krebel, 99).
Phuong phap 1-vs-alI xdy dung c md hinh SVM nhi phin, mo hinh thii t tach lop t (ldp duong) ra khdi cic ldp khic (am).
Phuang phap I-vs-1 xdy dyng c(c-l)/2 md hinh SVM nhi phdn, mfii mfi hinh tdch mfit cap 2 Idp.
Vile phdn ldp dua vdo binh chpn khoang cich din cac silu phang tfau dugc tir SVM nfai phan.
Nhu da chi ra trong thyc tl ciia pfain lop kho dii lieu inh rat Ion, hdng chuc ngdn ldp (Sanchez &
Perronnin, II), (Deng et al., 12), 1-vs-all thi don giin cho kit qud tdt cho vdn de phin ldp dnh.
Tuy nhien, khi ip dyng I-vs-all vao frong SGD de giii quylt vdn dl da ldp. chiing la lai gap mdt khd khan ldn, hudn luyen cic SGD nhi phdn trin tip dii lieu mat can bdng. Gia sii: tap dS lilu chung ta CO 100 Idp, thi khi sir dyng 1-vs-all, mfi hinfa SGD thii t tach ldp t (lop duong chi chiim klioing 1%) ra khdi cac Idp kfaac (Idp im chiim khoing 99%). SGD gap kfad khdn do sy mdt can bing.
SGD chi thuong cap nhat w frong (4) kfai loi xuat hien thuong Id phin tii thufic ldp am mi it khi ldm diiu do vdi Idp duong vi xdc suat lay mau rapt phan tir ldp duong khoang 1%. frong khi lop am la 99%. Mac dii diiu kfad khdn nay xiy ra nliung dp chinh xac tdng thi vdn la 99% frong khi khdng the tach ducrc ldp t ra klidi cac lop kfadc.
Bt giai quyet van de ndy. chiing tdi de xuat chiin luge xdy dyng balanced bagging eho timg md hinh SGD nfaj phdn dya trin liy miu giim \ a cap nhit frpng sd bdt ddi xiing. Huin luyln md hinh SGD nhi phdn tich Idp t (duong, thilu sfi)
kfadi cdc ldp kfadc (am, da sd). cin xay dyng k md hinh ca sd SGD nhu sau:
- Lay mau giim ldp im sao cho sd lupng phdn tir lop im bing vdi sfi phdn tii cua Idp duong.
Sir dyng tap mdu giam ldp im va dii lieu ciia ldp duang lara tap huan luyln mfi hinh co sd SGD.
Hudn luyln SGD nhi phin, chii y sii dung cfing thiic (4) voi cap nhit frpng sfi ldn hon kfai phdn lop sai dir lilu thufic Idp duang (thieu sd), vi frpng so nhfi hon khi phdn lop sai dir lieu thufic ldp im (da sfi).
Kit thiic, chiing ta tong hgp k mfi hinh co sd SGD thu dugc mfi hinh SGD nhi phan tdch ldp t lir cdc lop kfaic.
Can chii y rang, balanced bagging sii dung tap mdu giara cua ldp dm, giiip can bang phin bd dii lieu giiia 2 lop, khi nang liy ralu rapt phdn tii ciia ldp ducmg vi ldp im li gdn nhu nhau khi tien hdnh cap nhit w frong (4). Hon nua. lay mau giam ciia ldp im lam tdng khoang cich tdch ldp (duong, im). Dieu niy tao dilu kiln cho SGD hdi ty vdi tdc dp nhanh hon so vdi sii dung tap day dO. Chinh vi ly do dd ma gidi thuit MC-SGD sur dyng cac balanced bagging cd thi phdn Idp hieu qua t|.p du lieu da ldp.
4 KET QUA T H U C NGHIDM
Qt lien hanh danfa gid hieu qua ciia giii thuat MC-SGD cho phan ldp anii da lop, cfaiing tfii da cai dat giii thuat MC-SGD bang ngdn ngii lap frinfa C/C-H-. Ngoai ra, chiing tdi ciing can so sdnh MC- SGD vdi mpt giii thuit SVM chuan, dupe su dung phd bien frong cfing dfing miy hpc li LibSVM (Chang & Lin, 01). Tdt cd cdc giii thuit diu dugc thyc hien frln mpl may tinh ca nhan (Intel 3GHz, 2GB RAM) chay he dieu hanh Linux.
Tgp chi Kboa hgc Tnrang Dai hgc Cdn Tho
Hinb 6: Anb miu trong t^p ImageNet 10 Idp
Tap dii li|u tfayc ngfailm dugc Idy ve tir ImageNet (Deng et al., 12). Chiing tdi chpn lap gfim 6675 anh ciia 10 Idp (xem Hinh 6). Chimg tfii tich tap dii li|u ra thdnh tap huan luyln co 4450 inh va tip kiim thii cd chua 2225 dnh. De biiu diin dnh bang mfi hinh tiii tir tryc quan, chiing tdi sii dyng giai thugt phdt hiln dac frung cyc bp Hessian Affine ciia (Mikolajczyk & Schraid, 04) dl nit trich cic vec to rafi ti SIFT. Sau dd, thyc thi giii thudt *t-means (MacQueen. 67) de gom nhdm cac vec to md ti SIFT vio 50000 clusters tuong irng vfi'i 50000 tir tryc quan. Giai doan tiln xir ly tao ra hai tap (bang) dii lieu huan luyen, kilra' thir, tucmg ling vdi 4450 va 2225 phin tu, 50000 efaiiu va 10 ldp. Cfaung tfii cfi ging thay dfii sfi clusters (tir tryc quan tir 1000 din 100000) di tim cac kit qua tfayc nghiera tfit nhat. Cufii cimg. chiing tfii thu dugc dp chinh xdc fin dinfa vdi 50000 tir tryc quan.
Budc tiln xir ly nay Id duy nhdt cho hai giii thuat miy hpc ma chung tdi kilm thii frong bii vilt.
Chiing tdi sir dung tap hudn luyln dl xay dung md hinfa MC-SGD va SVM sir dyng hdra nhin phi tuyin RBF (SVM-RBF). Diiu chinh tham sd dya vio nghi thiic kilm fra eheo (hold-out) dupe dp dyng tren tap hudn luyln. MC-SGD sii dyng tham sd lambda = 0.1 (hang sfi quy tic, dimg diiu chinh dfi rfing 11 phin hogch), lap 7 chu ky (epoch) Ii hdi tu den kit qud tdt nfadt. Chiing tdi cd gang sir dyng ham nhan RBF ciia SVM (ham nhan RBF ciia hai diim dir lieu x„ x, Id K[i. j] = ^^^^ _ ^ | ^ _ _ , ^ | | ^ j . Gidi thudt SVM sii dung him nhan RBF (vdi
/=0.00001) vi hing sd quy tic c = 1000 (diiu chinh dp rdng Ie phan hoach \ a 161) cho kit qui tot nhat. Kit qui thu dugc frln tgp kiem thir nhu trinh biy trong Bing 1, Hinh 7.
Bdng 1: Ket qud phdn Idp anh Loll
0 1
7
3 4 5 6 7 g 9 Overall
MC-SGD (%) 99.16 95.73 79.17 95.77 89.27 96.34 89.11 85.89 85.11 77.37 88.72
SVM-RBF (%) 93.25 86.59 79.17 90.61 77.68 84.76 91.94 79.44 86.38 71.60 83.96
K&quaprtanlapanh
eooo 76,0D 70.00
eooo
Hinh 7: Ket qua phdn lop dnfa So sdnh kit qud cho thiy dugc MC-SGD phin ldp chinh xdc hon SVM-RBF. MC-SGD cho kit qua tdt nhdt 8 frong 10 ldp dii lilu inh vi cho dp chinh xic tdng the cao faon gdn 5% so vdi SVM- RBF. Hon niia, MC-SGD chi mat thdi gian hudn luyln la 2.20 giay frong khi SVM-RBF cin din 111.67 giay, hay ndi cich khic MC-SGD nhanh hon SVM-RBF 50 Idn.
Tap chi Khoa hgc Tnr&ng Dgi hgc Cdn Tho PhdnA: Kboa boc Tunhien. Cong nghe vd Moi truang. 29 (2013): /-/
Voi cac kit qui phin Idp niy. cfaiing tdi tin rang gidi thuit MC-SGD cho phep phin ldp failu qui dii lilu cd sd cfaieu Idn thu dugc tir budc bilu dien dnfa bang md fainfa tiii tir vi ddc tnmg kfafing ddi SIFT.
S KET LUAN VA DE XUAT
Y tudng ciia \ iec ting efaiiu frong bieu dien anh bang md hinh tiii tu dl cd tfal pfain lop don gian bdng md fainfa tuyln tinfa ma khfing can din mfi fainfa phi tuyin Id tiip cdn rit hieu qui. Khi sfi chiiu du lieu Ii nhd, chiing ta cin din md hinh phi tuyin de gidi quylt tdt van de phdn ldp. Tuy nhien. huin luyln mfi hinh pfai luyln cd dp phiic tap rdt cao so vdi md Wnh luyln tt'nh. Nhung ngugc lai, md hinfa tuyln tinfa tfai chi lim vile tdt frln tap dii lilu cd sd chjiu ldn. Chinh vi ly do dfi, tdng sd chiiu biiu dien dnh bang md hinfa nii tir de cd thi su dung mfi hinfa phin ldp tuyln tinfa la y tudng myel vdi cfao cd dp ehinh xdc vd thdi gian hudn luyln md hinh.
Chiing tdi da de xudt giii thuat phin ldp luyin tinh da Idp, gidm gradient ngiu nfailn MC-SGD. cho phep phan Idp hieu qui dur lieu cd sfi chiiu lon tfau dugc tir budc bilu diln inh. Kit qua tfayc nghiem frln tgp dii lilu thyc cho thiy giai thudt MC-SGD phin ldp nhanh. chinfa xac faon khi so sdnh vdi giii thuit may hoc vec ta hd frg (Support Vector Machines - SVM) sii dung faim nhin phi tuyin (Radial Basis Function - RBF).
Chiing tdi cung w a phat triln gidi thuat MC- SGD song song cho phep tang tdc qua frinh thyc tfai trin may tinh cd nhily bg xu ly, nhdm hay ludi mdy tt'nh. Trong tuang lai gin, cfaiing loi sii dung gidi thudt MC-SGD song song dl thyc hien phin ldp tap dCr lieu thir thich ImageNet-2012 cfi hon 1 frilu inh, 1000 lop khic nhau. Ben canh dd. chiing tfii cung mudn chung tdi dy dinh iing dyng phuang phip dt xuat vao vdn dl phdn Idp dnh, van bdn, video.
TAI LIEU THAM KHAO
1. Bosch. A., Zisserman, A., Munoz. X.; Scene clcissification via pLSA. In: Proceedings of the European Conference on Computer Vision, pp. 517-530 (2006).
2. Bottou, L., Bousquet, O.: The tradeoffs of large scale leaming. In Advances in Neural Information Processing Systems (20):l6l- 168(2008).
3. Chang. CC. Lin. C.J.: LIBSVM - a library for support vector machines (2001).
fattp://www.csie.ntu.edu.tw/-cjlin/libsvm
4. Deng, J., Berg, A., Saiheesh. S., Su. H..
Khosla, A., Li, F-F.: Large Scale Visual Recognition Challenge. (2012).
5. Guermeur, Y.: Svm multielasses, theorie et applications. Tfaese HDR (2007) 6. Krebel, U.: Pairwise classification and
support vector machines. Advances in Kernel Methods: Support Vector Leaming pp. 255-268 (1999).
7. LeCun, Y., Bottou L., Bengio, Y., Haf&ier.
P: Gradient-based leaming applied to document recognition. In Proceedings of the IEEE, 1998, pp. 2278-2324.
8. Lowe. D.: Distinctive image features from scale invariant keypoints. IntemationalJoumal of Computer Vision pp. 91-110 (2004).
9. MacQueen, J.: Some methods for classification and analysis of multivariate observations. Proceedings of 5th Berkeley Symposium on Mathematical Statistics and Probabilit>', Berkeley. University of California Press Vol. 1, pp. 281 -297 (1967).
lO.Mangasarian O.L.: Mathematical Programming for Support Vector Machines.
INRL\ Rocquencourt. France July 17 (2001).
11. Mikolajczyk, K., Schmld. C ; Scale and affine invariant interest point detectors.
International Joumal of Computer Vision 60(0,63-86(2004).
12. Piatt J.: Sequential Minimal Optimization:
A Fast Algorithm for Training Support Vector Machines. Microsoft Research Technical Report MSR-TR-98-14 (1998) 13.Quinlan, J.R.: C4.5. Programs for Machine
Leaming. Morgan Kaufmann. San Mateo.
CA(I993).
14. Sanchez. J., Perronnin, F.: High- dimensional signature compression for large scale image classification. In IEEE Conference on Computer Vision and Pattem Recognition pp. 1665-1672(2011).
15. Vapnik, V.: The Nature of Statistical Leaming Theory. Springer-Verlag (1995).
16. Viola, P.A.. Jones, M.J.; Rapid object detection using a boosted cascade of simple features. In IEEE Conference on Computer Vision and Pattern Recognition pp. 511-518 (2001).
I7.Zheng. H.. Daoudi, M.: Blocking aduh images based on statistical skin detection.
Electronic Letters on Computer Vision and Image Analysis 4(2y.l-\4 (2004).
