- - - - - -
Development of a Robust Algorithm for Transformation of a 3D Object Point onto a 2D Image Point for Linear Pushbroom Imagery
Taejung Kim, Dongseok Shin, and Young-Ran Lee
Abstract expressed in a rigorous mathematical form (Kim, 2000). Further
A powe$ul and robust algorithm for the Indirect Method, i.e., studies are required for the understanding of the geometry of the transformation of a 30 object point onto a 2D image point linearpushbroomimages.
for linear pushbroom imagery, is proposed. This algorithm This paper addresses the problem of rectification of linear solves the transformation iteratively with an initial estimate pushbroom images so that the images can be geometrically ref- of the 2 0 image point coordinates. However, this algorithm erenced. There can be two ways to solve this problem. The first does not require any sophisticated procedures to determine a one is referred to as the Direct Method, which solves the prob-
"good" initial estimate and it always converges to the correct lem by projecting an image point in two-dimensional (zD)
solution. This algorithm works using the following procedures: image coordinates onto an object point in three-dimensional first, with an (random) initial estimate of the 2 0 image point (3D) object coordinates. A technique called Ray-Tracing coordinates, calculate the attitude of the camera platform; (O'Neilland Dowman, 1988) was developed to solve the prob- second, with the given attitude, calculate the position of the lem by the Direct Method. The other one is referred to as the camera platform and the 2 0 image point; and third, update Indirect Method, which solves the problem the other way the estimate with the calculated 2D image point coordinates around, i.e-9 by projecting a 3D object point onto a zD image and then go back to the first procedure and continue iteration point. It is known that the Indirect Method has many advan- until the estimated and calculated image point coordinates tages overthe Direct Method (Mayrand H e i ~ k e , 1988). In Par- converge. Results of the experiment show that this algorithm ticular~ it reduce sthe processing time and amount of memory converges very fast even when the initial estimate has a required for image resampling.
huge error. However, a robust numerical solution of the Indirect
Method for linear pushbroom images has so far not been devel-
Introduction oped. Chen and Lee (1993) proposed one solution based on the
Due to the success of recent commercial remote sensing satel- Newton-Ra~hson method (Cheney and Kincaid, 1994). How- lite programs, linear pushbroom images taken from such satel- ever, the Newton-Ra~hson method works only within the lites are more and more widely used in various application region where the equation to be solved varies monotonically areas. Many new missions are being planned or pursued (Cheney and Kincaid, 1994) and, hence, it is very sensitive to worldwide. In particular, 1-m resolution remote the initial value. In some applications, this method may work sensing satellites will offer a large number of new applications but in others (in particular, for airborne linear pushbroom and their images can be agood alternative to perspective images images), it may not be easy to choose ''appropriate" initial Val- taken from an aircraft. ues. Due to this reason, rectification of linear pushbroom
In order to utilize linear pushbroom images in practical imageshasbeen done by theDirectMethod3 despite the many applications, it is important to understand the geometry of lin- advantages of the Indirect Method.
ear pushbroom images in relation to the object surface. This We propose a new powerful and robust method, which we task is, however, not as trivial as that for perspective images refer to as Back-Tracing, to solve the problem of projecting a 3D because the platform (or satellite) is in motion during imaging object point onto a zD image point. This method works because and hence there is one perspective center per one scan line. In we assume (not the equation to be solved, but) the attitude and fact, the geometry of linear pushbroom images is not yet fully position of the platform to be monotonically varying.
understood. Although several models to relate the image coor- In the next section, we briefly introduce the geometric dinates to the object coordinates had been developed (Konecny model of linear pushbroom images that we used here. The et al., 1987; Gugan and Dowman, 1988; Orun and Natarajan, Chen and Lee's method (Chen and Lee, 1993) will be mentioned 1994), it has only been recently, for example, that the epipolar- for comparison with the Back-Tracing method. In the following ity of a linear pushbroom stereo image pair was discovered and section, the Back-Tracing method and its principle will be
explained. Finally, the experiment results will be given to Satellite Technology Research Center, Korea Advanced Insti-
tute of Science and Technology, 373-1 Kusung-Dong, Yusung, Taejon 305-701, Korea ([tjkimldshinlyllee]@krsc.kaist.ac.kr), D. Shin is presently with SaTReCi Co. Ltd., 929 Dunsan-Dong,
Seo-Gu, Taejon 302-120, Korea. Photogrammetric Engineering & Remote Sensing Vol. 67, No. 4, April 2001, pp. 449-452.
Ms. Lee is presently with the Department of Civil and Environ-
mental Engineering and Geodetic Science, The Ohio State 0099-lllZ/Ol/6704-449$3.00/0 University, 2070 Neil Ave., 470 Hitchcock Hall, Columbus, 0 2001 American Society for Photogrammetry
OH 43210. and Remote Sensing
PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING Apr1l2001 449
show that the Back-tracing method works with almost any ini- tial values and rapidly converges to the correct solution.
A
Geometric Model of Linear Pushbroom Images and Chen and Lee's MethodThere have been several models proposed to relate 2D linear pushbroom image coordinates and 3D object coordinates (Konecny et al., 1987; Gugan and Dowman, 1988; Orun and Natarajan, 1994). In this paper the model proposed by Orun and Natarajan (1994) will be used. However, the Back-Tracing method proposed here can be applied to any models based on the collinearity equations.
The collinearity equations for perspective images can be modified for linear pushbroom images as follows:
Back-Tracing Algorithm
(1) By substituting Equation 3 into Equation 4, it is possible to re- write Equation 4 as follows:
where
X, Y , Z are the coordinates of an object point;
Xs, Ys, Zs are the position of the platform at the instant of imaging;
ril,
. . .
, r33 are the elements of the rotation matrix deter- mined by the attitude angles KS, q5s, us of the platform at the instant of imaging;f is the focal length of the camera; and x, yare the coordinates of an image point.
Orun and Natarajan (1994) modeled the position (XS, Ys, Zs) and attitude ( K ~ ,
& ,
us) of the platform as polynomial func- tions of the image coordinate1 x: i.e.,method to refine the image coordinate through iteration. It is well known that the Newton-Raphson method works only within the region where the equation to be solved varies mono- tonically and, hence, this method is very sensitive to the initial value (Cheney and Kincaid, 1994). However, it may not be easy to find monotonic regions of Equation 4 in advance and to choose an initial value xto lie within such regions. For satellite linear pushbroom images, this problem may not be severe because we can have a stable platform motion and accurate on- board time informati~n.~ For airborne images, this problem may occur.
In the next section, we propose a new powerful and robust algorithm to solve the Indirect Method, which works whether or not Equation 4 varies monotonically.
4s = (Po
where (X,, Yo, Z,) and ( K ~ , (Po, w,) are the position and attitude of the platform at the scene center, respectively. Camera model- ing is the procedure to determine the unknown parameters in Equations 1 and 2 using a number of known image coordinates (x, y) and known object coordinates (X, Y, Z).
As Chen and Lee (1993) have pointed out, the problem of the Inverse Method is to find the image coordinate x satisfying the following equation:
Unlike perspective images, the rotation elements rll, rzl, r3, and the position of the platform (Xs , Ys , Zs) are not constants but functions of x in the case of linear pushbroom images.
Hence, the above equation is a non-linear equation of the image coordinate x and it cannot be solved analytically.
Chen and Lee proposed an algorithm to solve the above equation numerically. They first assumed an initial value for the image coordinate x and they applied the Newton-Raphson
lThe use of the image coordinate x is an approximation of the time of imaging.
450 April ZOO1
Note that the rotation elements r,,, rZl, ~ 3 , are non-linear func- tions of x. The problem of solving the Indirect Method is to find the solutionxsatisfying f(x) = 0. We propose the followingpro- cedures to find such solution x:
(1) Assume an initid value x,, and calculate3 the attitude of the platform KS, (bS, us at x,. Using the attitude parameters, calcu- late I-,,, r,,, r3, in Equation 5.
(2) Consider rl,, rZl, r,, as constants and Equation 5 as a second- order polynomial in x and calculate x, satisfying Axl) = 0.
(3) Update xo with x, and iterate procedures (11 and (2) until x, and xl converge.
Once the image coordinate xis found, the image coordinate y can be calculated using Equation 2. The principle of this Back-Tracing algorithm can be explained by examining Figures 1 and 2.
Figure 1 shows the flight path of the platform, imaging tracks on the object surface at a given position, and an object point A. In procedure (1) above, the attitude of the platform and the elements of the rotation matrix are calculated assuming an initial value x,. This corresponds to defining the camera axis at x, as shown in the figure. The imaging track at x,, does not con- tain the object point A because X, is not the solution we are seeking.
In procedure (21, the elements of the rotation matrix are considered as constants and xl is found by solving the second- order polynomial in x. This corresponds to assuming that the attitude of the platform at x, applies throughout the flight path and shifting the position of the platform so that the imaging track on the object surface contains the object point A.
However, as Figure 2 illustrates, the true attitude of the platform at xl is different from the attitude at x, and the true imaging track atx, does not contain the object point A. In proce- dure (3), x, is updated with x1 and procedures (1) and (2) are iter- ated until xo and x, converge. This corresponds to updating the attitude of the platform at x1 with the true attitude and iterating procedures (1) and (2) until the imaging track on the object sur- face at xl assuming the attitude at x, and the true imaging track at xl converge and both contain the object point A.
ZHowever, we observed several times that the image coordinate x did not converge even with satellite images.
3Because a camera model must have been established using ground control points, the attitude of the platform can be "calculated" at a given point x,.
PHOTOGRAMMETRIC ENGINEERING 81 REMOTE SENSING
Flight path of the platform
Boundaries of object
Figure 1. Geometry of linear pushbroom images and imaging tracks on the object surface.
Discussions
than is x,. However, this limitation is not severe because theThere is one limitation of the Back-lkacing algorithm that we motion of the platform is generally along the orbital path and propose here. This algorithm works within the portion of the the attitude is kept stable within a small error range while flight path where the position and attitude of the platform vary imaging- This is a major advantage over Chen and Lee's alga- monotonically. We can deduce this limitation easily from Fig- rithm which works within the region where the non-linear mes 1 2. If the attitude and flight path varies non-monoton- equation itself in Equation 5 varies monotonically.
ically, we cannot guarantee that xl is closer to the right solution Table 1 shows the results of the Back-'lkacing method pro-
Assumed C B m e r a axis at XI
(whichisthetrueoamnaaxisatXo)
-.. "..,
True imaging track on object surface at Xl
Object Point A Imaging track on object
surface at Xl assuming the attitude of the platform at '0
.
Figure 2. The difference between the assumed attitude of the platform and the true attitude at x, and the imaging track on the object surface in each case.
PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING April 2001 451
Iteration Solution by the Initial Values Number Back-Tracing Method Truth - 500 sec
Truth - 100 sec Truth - 5 sec Truth - 1 sec
Truth - 50*0.001504 sec Truth - 10*0.001504 sec Truth
Truth
+
10*0.001504 sec l h ~ t h+
50*0.001504 sec Truth+
1 secTruth
+
5 secT h t h
+
100 sec Truth+
500 secposed here. We used x as the imaging time of a particular scan line. At the scene center, we define x = 0 and, at a particular scan line r, we calculated x = (r - 3000) X 1.504msec, where
3000 is the scan line number at the scene center and 1.504msec is the scanning rate of the linear pushbroom image we used for experiments (SPOT PAN). Before applying the Back- Tracing method, we set up a camera model with ground control points acquired by differential GPS surveying.
We chose one ground control point to test the Back-Tracing method. Assuming that the x coordinate of the test GCP as truth (-2.004140139 sec), we applied the Back-Tracing algorithm with several xo's whose values deviated from the truth at differ- ent magnitudes. We set iteration stop when the difference between xo andx, became less than 0.0104msec (or 0.01 pixel).
As Table 1 shows, the Back-Tracing algorithm converged to almost identical solutions regardless of initial values. In par- ticular, this algorithm worked with an initial value whose error was as large as 500 seconds (or more than 332,000 pixels). The iteration number required was three when an initial value was close to the truth, i.e., within 5 seconds (or 3324 pixels) and not more than six even with an initial value with a large error. We concluded that the Back-Tracing algorithm converged very fast.
Conclusions
In this paper, we proposed a very powerful and robust algorithm to solve the problem of the Indirect Method,
i.e., the transformation of a 3D object point into a 2D image point. We provided the principle of the proposed algo- rithm and proved with experiments that the proposed algorithm worked.
The contribution of this paper is the provision of a numerical solution for the Indirect Method, which has been known to be unsolvable (or only partially solvable by an algorithm based on the Newton-Raphson method). The theme of our algorithm is the decomposition of the prob- lem of the Indirect Method into the problem of solving a linear equation assuming the attitude of the platform is constant. This algorithm works independent of the choice of initial value and rapidly converges to the correct solution.
The algorithm proposed here should be applicable to wide application areas using linear pushbroom images, including rectification, ortho-correction, and generation of digital elevation models.
Read PE&RS on-line!
Excerpts of PEbRS are now available on-line..
Plus, stay tuned for many more advances to the
ASPRS web site.
www. as
References
Chen, L-C, and L-H Lee, 1993. Rigorous Generation of Digital Orthopho- tos from SPOT Images, Photogrammetric Engineering & Remote Sensing, 59(5):655-661.
Gugan, D.J., and I.J. Dowman, 1988. Accuracy and Completeness of Topographic Mapping from SPOT Imagery, Photogmmmetn'c Record, 12(72):787-796.
Kincaid, D., and W. Cheney, 1996. Numerical Analysis: Mathematics of Scientific Computing, Second Edition, BrooksICole, Pacific Grove, California, 804 p.
Kim, T., 2000. A Study on the Epipolarity of Linear Pushbroom Images, Photogrammetric Engineering & Remote Sensing, 66(8):961-966.
Konecny, G., P. Lohrnann, H. Engel, and E. Kruck, 1987. Evaluation of SPOT Imagery o n Analytical Photogrammetric Instruments, Pho- togrammetric Engineering b Remote Sensing, 53(9):1223-1230.
Mayr, W., and C. Heipke, 1988. A Contribution to Digital Orthophoto Generation, International Archives of Photogrammetry and Remote Sensing, 27(Bll):IV 430-IV 439.
O'Neill, M.A., and 1.1. Dowman, 1988. The Generation of Epipolar Synthetic Stereo Mates for SPOT Images Using a DEM, Interna- tional Archives of Photogrammetry and Remote Sensing, Kyoto, Japan, 27(B3):587-598.
Orun, A.B., and K. Natarajan, 1994. A Modified Bundle Adjustment Software for SPOT Imagery and Photography: Tradeoff, Photo- grammetric Engineering & Remote Sensing, 60(12):1431-1437.
(Received 17 August 1999; accepted 28 March 2000; revised 09 May 20001
Grids
&Datums Abstracts Software Reviews
Calendar Notices Classifieds Book Reviews and many more
p r s org
PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING