First, I would like to thank my advisor, Jim Arvo, for being a constant source of ideas and inspiration, and for creating many of the illustrations in the text. I would like to thank all the students of the Computer Science Department for support over the years, and especially the members of the Graphics Group.
Motivation
The hierarchy from which the examples in this thesis are derived is shown in the lower left of Figure 1.2 and reproduced in Figure 5.1. Suppose we have a collection of objects such as cylinders, bowls, and spoons that we wish to represent and which are effectively modeled by our chosen hierarchy.
Benefits of the Proposed Method
The algorithm takes distance data (in the form of a point cloud) and a generative model hierarchy as input. An appropriate generative model is then selected and the parameters optimized to provide an accurate and concise description that can be edited.
Related Work
However, the simplified mesh will still contain the same primitive – the triangle – and will still not represent many models accurately. In addition, our chapter on optimization of generative models discusses some improvements over the objective functions used in Hoppe et al.[l6].
Limitations of our Approach
Hierarchical Structured Light
We include the object to be scanned, the projector, the camera, the checkerboard for calibration, the monitor and the computer that displays the point clouds of the object. A sequence of alternating black and white bars are projected as a gray code onto the object by the projector.
Desktop Range Imaging
This is because there is little contrast between views obtained by projecting dark and light stripes, or between views obtained when the object is in shadow versus being illuminated. The object to be scanned must be rigid so that it can be easily rotated so that it can be viewed from multiple viewpoints.
Generative Modeling
To create a generative model, the user must know the surface description, in other words, the user must specify the precise symbolic description of the model. We note that a major obstacle in generative modeling is requiring the user to specify the function f in Equation 2.1, often using complex mathematical operations.
Fitting of Parametric Models using Optimization
Model Hierarchy
The model hierarchy consists of a number of levels that correspond to the complexity of the model as measured by the number of curves it consists of. For our class of models, level 0 consists of a constant thickness “box” with only 3 parameters for the length, width and depth. For example, the box is the parent, while the refined depth model is the child.
While the parameters for the curves may be constantly changing, there may also be parameters that are changing. In this case, we can use one of these values in the recognition phase, but before the final selection and optimization of the model, we choose the value that leads to the smallest error. For example, in the tree shown in Figure 1.2 and discussed in more detail in Chapter 5, each node can be split to correspond to a plane or circular cross section.
In practice, we have found it convenient to use the simple rectangular (straight) section for the recognition process. The branch for the appropriate diameter is the lower part of the tree in figure 1.2.
Estimation of Co-ordinate Axes
For completeness, the results show an example (the bowl) where we actually split each node.
Parameter Estimation
We emphasize that model-dependent estimates can be quite crude, as they merely provide an initial guess for the global optimization phase.
Algorithm Description
Since the model can be an arbitrary function of the parametric curves that make it up, we use a general, unrestricted, nonlinear minimizer. We will denote the i-th point from the obtained range data by (zri, yri) and the i-th point from the tessellation of the model by (xmi, ymi) ⋅ (xmi, ymi) is assumed to be randomly smooth in the parameters of the model. Integration range: Where Yrmax(x) and Yrmin(x), or Ymmax(x) and Ymmin(x) are undefined (that is, if the points from range data or model do not extend to a particular region), the integrand( s) is (are) defined as 0.
2D Objective Function: We will use a simple objective function, illustrated in Figure 4-1: the area not common to the two projections. For Cp continuity, f must be Cp continuous and satisfy f(u) = u when u ≥ t where t is the "support" of f - the region where it differs from the traditional value. An inspection of the lower center portion of Figure 4-2 will show that this construction requires that there be regions where f will be less than that for the traditional maximum.
Note that although our specific model is symmetric about the X-axis, the range data is not similarly constrained, which accounts for most of the final error. On the left side of this figure, for a large region, almost all points map to the same region of extent data, and a similar effect is seen in cases where the model is poorly fitted. Also note that our comparisons are based on the geometric properties of the generative model and area data and are general.
Because of the integral below, ym⇓(xm) and ym⇑(xm) Cp are continuous in the model parameters if the kernel of Cp is continuous.
Curve-Fitting
In the right figure, we show the silhouette with x marks, and the estimated curve with a solid line with control points marked with circles. Since the model is symmetrical about the X-axis, control points are shown for one side only, and the other side of the curve is shown for illustrative purposes only and is derived from symmetry. Note that there is a high density of control points in areas of high curvature with a correspondingly low density in areas of low curvature.
Combining Curve-Fitting and Optimization
In the left image, the range data is projected onto the X-Y plane and a (noisy) silhouette is extracted. Coordinate axes: The directions of the coordinate axes are found by computing the principal axes of the data using the eigenvectors of the covariance matrix, as done for the computation of delimiters in [13], or by considering the axes along which the range data have the largest extent—in this approach first we find the longest axis by optimization and then the longest axis perpendicular to the first. In general, we can expect the use of the covariance matrix to be more consistent - to enter a uniformly sampled cube, the axis axes of the cube were created.
However, for elongated objects where a significant amount of data is missing, using the longest axis is more robust. Note that we expect the input range data to contain only points of the object of interest, not the environment—stray points can seriously affect the determination of the longest axis. Labeling the X and Z axes requires that the silhouette of the series data in the X-Z plane be parallel to the X axis (as would be the case for the "box" model we are originally trying to fit has).
To avoid noise, we only consider the regions where the depth of the sequence data is greatest to calculate this silhouette. Since the thickness is typically small, and there are poor clues about it in the extent data, we do not optimize over it, but instead treat it as a constant value derived from looking at the projection in the Y-Z plane of the acquired data for a suitable X as illustrated in figure 5.3.
Shapes Modeled
Generalized cylinder (variable form): The projection of the box on the X-Y plane is a rectangle with dimensions l and w. This is the essence of going down one level in the tree of generative models: a curve has been added for better representation. This curve (assumed to be symmetrical about the X axis) is obtained by considering the silhouette of the range data projection on the X-Y plane.
The curve fitting part of section 3 shows an example of this projection and the derived curve. We implement this by placing a triple knot at the endpoints with P[1] = 0, and also ensuring that the next control point at both ends has P[0] the same as the previous 3 to ensure that the curve is perpendicular on the X-axis at the extremes. Variable depth: Instead of the depth being constant, it can also be made a curve like the shape.
In case the derivative is discontinuous here, a doubled node must be placed at the zero crossing. We give below the representation for the complete spoon model (replacing ST with ARC and adding H for the bending parameter so that H[0] = P[0] = H[0]).
Simplified Shapes
We use the 3D fit error function φ defined by equation 4.12 using the simple distance norm after scaling the range data so that the major axis lies in the range -1 to +1. We show the fitting error data in Figure 6.2 after splitting each node corresponding to straight and circular. In the bowl tree on the right, each node is split according to straight and circular cross sections and a choice is made at each level.
Motion: In Figure 6.4, we show the fitting of a generative model to a single image of a range of a spoon obtained using the method of Bouguet and Perona [4]. Cup: Figure 6.5 shows the robustness of our approach even when the model does not fit the range data well. Range data segmentation (equivalent to adding a join operator to our model hierarchy) and a small addition to our model tree (adding a full rectangular cross section instead of just a semi-rectangular cross section or right) allows us to model the entire cup quite accurately.
Correct and efficient methods of curve representation, especially in the presence of discontinuities, are also a direction for future research. In the future, we also want the system to automate certain tasks that currently require user interaction.
