isprsarchives XL 1 W4 351 2015

(1)

Automated DSM extraction from UAV images and Performance Analysis

Sooahm Rheea, Taejung Kimb a

3DLabs Co., Ltd., Incheon, Korea – [email protected] b

Dept. of Geoinformatic Engineering, Inha University, Incheon, Korea – [email protected]

Commission I ICWG I/Vb

KEY WORDS: DSM, stereo matching, epipolar lines, UAV, Photogrammetry

ABSTRACT:

As t echnology evolves, unm anned aer ial vehicles ( UAVs) im ager y is being used fr om sim ple applicat ions such as im age acquisit ion t o com plicat ed applicat ions such as 3D spat ial infor m at ion ext r act ion. Spat ial infor m at ion is usually pr ovided in t he for m of a DSM or point cloud. I t is im por t ant t o gener at e very dense t ie point s aut om at ically fr om st ereo im ages. I n t his paper , we t r ied t o apply st er eo im age- based m at ching t echnique developed for sat ellit e/ aer ial im ages t o UAV im ages, pr opose pr ocessing st eps for aut om at ed DSM gener at ion and t o analyse t he possibilit y of DSM gener at ion. For DSM gener at ion fr om UAV im ages, fir st ly, ext er ior or ient at ion par am et er s ( EOPs) for each dat aset wer e adj ust ed. Secondly, opt im um m at ching pair s w er e det er m ined. Thir dly, st er eo im age m at ching was per for m ed wit h each pair . Developed m at ching algorit hm is based on gr ey- lev el cor r elat ion on pixels applied along epipolar lines. Finally, t he ext r act ed m at ch r esult s wer e unit ed wit h one r esult and t he final DSM was m ade. Gener at ed DSM was com par ed wit h a r efer ence DSM fr om Lidar . Over all accur acy was 1.5m in NMAD. How ev er, sev eral problem s have t o be solved in fut ure, including obt aining precise EOPs, handling occlusion and im age blurring problem s. More effect ive int erpolat ion t echnique needs t o b e dev eloped in t he fut ure.

1. INTRODUCTION

As t echnology evolves, unm anned aer ial vehicles ( UAVs) im ager y is being used fr om sim ple applicat ions such as im age acquisit ion t o com plicat ed applicat ions such as 3D spat ial infor m at ion ext r act ion. Spat ial infor m at ion is usually pr ovided in t he for m of a DSM or point cloud. I t is im por t ant t o gener at e ver y dense t ie point s aut om at ically fr om st er eo im ages. Many im age m at ching t echniques have been pr oposed for t his pur pose ( Haala and Rot her m el 2012; Jeong and Kim , 2014 ) . However , DSM gener at ion fr om UAV im ages is st ill challenging. Unlike sat ellit e and aer ial im ages, UAV im ages possess only sm all cover age. UAV im ages wit h cent im et r e-level ground sam pling dist ance cont ain m any t ext ur e- less feat ur es such as r oofs and pavem ent s, m oving obj ect s and r epet it ive pat t er ns.

I n t his paper , we t r ied t o apply st er eo im age-based m at ching t echnique developed for sat ellit e/ aer ial im ages t o UAV im ages, pr opose pr ocessing st eps for aut om at ed DSM gener at ion and t o analyse t he possibilit y of DSM gener at ion.

2. DATASET

To obtain the test dataset, we used the DJI’s Spread Wings UAV which is hexa rotor type (Figure 1).

Figure 1. Spread Wings UAV

The im age dat a set was acquir ed fr om SONY’s DSLR I LCE- 7R cam er a m odel wit h a focal lengt h of 35m m and a det ect or size of 4.88967 x 4.88967 um . The flying height of t est ar ea was 180m , which r esult in a gr ound sam pling dist ance ( GSD) was 2.5cm .

I n t his st udy, we acquir ed im ages including t he t ext ur e- less ar eas, such as t he br idge and bare gr ound. The acquir ed im age size is 4800x3200

The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XL-1/W4, 2015 International Conference on Unmanned Aerial Vehicles in Geomatics, 30 Aug–02 Sep 2015, Toronto, Canada

This contribution has been peer-reviewed.

(2)

pixels. Figur e 2 show a few im ages of dat aset fr om UAV.

Figure 2. A few images from the UAV dataset

3. STEREO MATCHING

3.1 Aerial Triangulation

For DSM gener at ion fr om UAV im ages, fir st ly, ext er ior or ient at ion par am et er s ( EOPs) for each dat aset wer e adj ust ed. I m age was acquir ed by t he UAV has posit ion and or ient at ion infor m at ion obt ained fr om on- boar d sensor s. For UAVs, sensor accur acy was lower t han t hose m ount ed on t he aer ial or sat ellit e im ages. The st abilit y of t he plat for m was also low. Ther efor e init ial EOPs caused ser ious m at ch er r or s and poor r esult s. We updat ed EOPs using som e cont r ol point s aut om at ically gener at ed

.

We com par ed each m at ching r esult bet ween befor e and aft er applicat ion of t he AT pr ocess wit h sam e m at ching cir cum st ances.

Table 1 show t he differ ence bet ween befor e and aft er applicat ion of t he AT pr ocess.

Before AT Aft er AT Diff. Bx ( m ) 152285.167 152285.383 0.216

By ( m ) 357503.229 357502.315 0.914

Bz( m ) 141.448 140.914 0.534

w ( deg.) - 1.791 - 2.236 0.445

p( deg.) - 0.417 - 1.295 0.878

k ( deg.) 143.094 143.172 0.078

Table 1. EO parameters sample

Figur e 3 shows ext r act ed point clouds under t he sam e m at ching condit ions using t he EOPs befor e and aft er AT pr ocess. I n t hese r esult s, it is cer t ain t hat m or e m at ching success point s wer e gener at ed when applying AT, and blunder ar eas wer e decr eased r elat ively.

Figure 3. UAV Stereo DSM (Left: using initial EO parameters, Right: using adjusted EO parameters)

3.2 ROI Setting

For gener at ing a DSM, we aim ed t o use all of t he im ages which lay wit hin t he r egion of int er est ( ROI ) by finding opt im al st er eo pair fr om all UAV im ages.

St er eo cover age bet ween t wo im ages and conver gence angles wer e checked for t his pur pose. I f t he st er eo cover age bet ween t wo im ages was t oo sm all or t he conver gence angles wer e t oo lar ge or t oo sm all, t he pairs wer e not consider ed.

3.3 Stereo matching

I f st er eo pair s ar e defined, st er eo im age m at ching was per for m ed wit h each pair ( Rhee et al., 2013) . Developed m at ching algorit hm is based on gr ey-level cor r elat ion on pixels applied along epipolar lines. Accur at e m at ched point s wer e ext r act ed by applying gr ey- level cor r elat ion r epet it ively wit h var ious cor r elat ion windows and select ing best candidat es by r elaxat ion. And by using m ult iple cor r elat ion windows, m or e accur at e im age m at ching t han t he exist ing ar ea based m at ching t echnique is possible.

I n addit ion, by using t he pyr am id- based t echnique, t he m at ching st abilit y was enhanced. Once st er eo im age m at ching was com plet ed for all m at ching pair s, a DSM fr om each pair was ext r act ed. The m axim um pat ch size was set t o 25x25 pixel for exper im ent s. The or iginal im ages wer e down- sam pled t o 1 / 8, 1 / 4, and 1 / 2 of t he or iginal size for pyr am id m at ching. Figur e 4 shows a few exam ples of DSMs fr om successfully m at ched pair s.

Figure 4. Examples of DSMs from successfully matched stereo pair

(3)

Finally, t he ext r act ed m at ch r esult s wer e unit ed wit h one r esult and t he final DSM was m ade. At t his t im e, t he line- infor m at ion like t he edge in t he im age was ut ilized bet ween DSM and it set t o t he boundar y line. And t he int er polat ion wor k was applied. Mat ching r esult s and accur acy analysis ar e r epor t ed in t he next sect ion.

4. DSM ACCURCY ANALYSIS

I n t his sect ion, we discuss st er eo m at ching per for m ance and DSM accur acy for t he sam ple dat aset . For exper im ent s, we used algor it hm s for t iepoint ext r act ion, AT and st er eo m at ching developed in- house as shown in Figur e 5.

Figure 5. Layout of SW tool used for experiments

Figur e 6 shows a DSM m osaic im age which was gener at ed fr om DSMs of individual pair s. I n t his figur e we can obser ve som e height differ ence on boundar y of individual DSMs. Never t heless, we confir m ed t hr ough t he ex per im ent al r esult s t hat a DSM m osaic could be gener at ed fr om piecewise DSMs gener at ed by individual pair s. A final DSM at 5cm gr id spacing was gener at ed. This DSM was com par ed wit h a r efer ence DSM at 50cm gr id spacing obt ained fr om air bor ne Lidar ( Figur e 7) .

Figure 6. A DSM generated over entire ROI

For t he quant it at ive accur acy analysis, RMSE ( Root Mean Squar e Er r or ) and NMAD ( Nor m alized

Median absolut e deviat ion) was calculat ed ( Table 2) .

No. of grids

compared

RMSE

(m)

NMAD

(m)

68.3

Quantile

95

Quantile

3197970 3.0553 1.5072 3.1917 4.9818

Table 2. Accuracy assessment of UAV stereo images

The Differ ence Map bet ween gener at ed DSM and LI DAR dat a was m ade in or der t o analyse t he cause for t he er r or ( Figur e 8) . The num er ical value of t he Differ ence Map was indicat ed wit h t he absolut e value of t he differ ence.

Figure 7. 50cm LIDAR DSM of the same area as figure 6

Figure 8. Difference map for target area

The t able and differ ence m ap showed r at her lar ge er r or s com par ed t o t he gr ound sam pling dist ance of t he UAV im ages. The m aj or fact or s for t his er r or could be inaccur acy of EOPs used and t he r esolut ion differ ence bet ween t he gener at ed DSM wit h t he LI DAR DSM used as t he r efer ence. Besides, lar ge er r or s occur r ed on t he boundar y of t he br idge, due occlusions caused by br idges.

I n or der t o analyze t he m aj or causes of t he er r or in dept h, t he coor dinat es of t he st er eo im age cor r esponding t o t he coor dinat e of t he DSM was r epor t ed ( figur e 9- 11) . Thr ough t he each r esult s, we found t hat blunder s w er e fr om r oads wit h ver y hom ogenous t ext ur e and t he sm all and t hin st r uct ur e such as t he st r eet light poles ( figur e 10) . Holes on r oad wer e m ainly due t o m oving obj ect s ( Figur e 11) . And im age blurring caused som e

(4)

problem s. I n part icular t he degr ee of blurring w as different fr om im ages t o im ages.

Figure 9. Matching success area (Left: DSM, Centre: left image,

Right: right image)

Figure 10. The blunder due to structure (Left: DSM, Centre: left image, Right: right image)

Figure 11. The blunder due to moving object (Left: DSM, Centre: left image, Right: right image)

3D point clouds wer e m apped wit h t he RGB values at t ached t o each point fr om t est dat aset and t he r esult is shown ingur e 12. For t his r esult we did not apply post - pr ocessing t echniques, such as point filt er ing or int er polat ion.

Figure 12. Point cloud generated over entire ROI with RGB values

The m any holes happened on t he m oving obj ect and blur r ing r egion. To m ake a m or e accur at e DSM, it is necessar y t o exper im ent wit h m or e im ages. And effect ive int er polat ion and noise

r educt ion t echnique needs t o be developed in t he fut ur e.

5. RESULT

I n t his paper , we invest igat ed t he possibilit y of pr ecise DSM gener at ion fr om UAV im ages. We t r ied t o apply st er eo im age- based m at ching t echnique developed for sat ellit e/ aer ial im ages t o UAV im ages. We t est ed st er eo m at ching per for m ance over UAV im age pairs and gener at ion of one m osaic DSM over t he ent ir e ROI . The accur acy w as ver ified com par ed wit h a DSM der ived fr om LI DAR dat a. Over all accur acy was ar ound 1.5m of NMAD. Pr oposed m et hod showed high per for m ance even over m any hom ogeneous ar eas.

However , sever al pr oblem s have t o be solved in fut ur e. Pr ecise EOPs wer e one of key issues. I t was ver y difficult t o obt ain pr ecise EOPs t o t he level of pr ecise geom et ric analysis. I n som e im age pair s, t he noise and m ism at ched ar ea wer e occur r ed due t o inaccur at e EOPs. Dur ing t he m osaic DSM pr oduct ion, som e height differ ence was founded in t he boundar y ar ea bet ween t wo individual DSMs. I m age blur r ing caused som e pr oblem s. I n par t icular t he degr ee of blur r ing was differ ent fr om im ages t o im ages. And effect ive int er polat ion and r elaxat ion t echnique needs t o be developed in t he fut ur e.

REFERENCES

Haala, N., & Rothermel, M., 2012. Dense multiple stereo matching of highly overlapping UAV imagery. ISPRS-International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 39, B1.

Jeong, J., and Kim, T., 2014, Analysis of Dual-Sensor Stereo Geometry and Its Positioning Accuracy, Photogrammetric Engineering and Remote Sensing, 80(5):653-661

Rhee, S., Kim, T., and Park, W., 2013. Precision DEM extraction using pseudo GCPs with ALOS/PRISM data, Proceedings of International Symposium on Remote Sensing, Chiba, Japan, 15-17 May 2013