Introduction
Motivation
An analog video is essentially a discretized version in the time dimension and in one of the two spatial dimensions. Video coding schemes, such as MPEG-1, MPEG-2, H.261, and H.263, were introduced with the goal of standardizing video compression algorithms to meet the bandwidth requirements of the applications they were designed to support. The Systen1S part of MPEG-4 deals with the description of the relationships between the audiovisual components that make up a scene.
To enable efficient implementations of the stru1dard, subsets of MPEG-4 Systen1S, Visual, ru1d Audio tools have been defined to be used by Cru1 for specific applications. However, it should be noted that a single rectangular area fonnat is treated as a special case of an arbitrarily shaped VOP ideology, where the entire frame is treated as a single VOP. The shape, motion, and texture information of VOPs belonging to the same VO are encoded and transferred or encoded into a separate video object layer (VOL).
In addition, the relevant information needed to identify each of the VOLs—and how the various VOLs are composed at the receiver to reconstruct the entire original sequence—is also included in the bitstream. This allows the separate decoding of each VOP and the required flexible manipulation of the video sequence as shown in Figure 1-1.
![Figure 1-1 : Video Obj ect Plan e Generation [Siko, 1997]](https://thumb-ap.123doks.com/thumbv2/pubpdfnet/10643193.0/13.846.219.731.670.888/figure-video-obj-ect-plan-generation-siko-1997.webp)
Investigated approach
The top level of the hierarchy consists of three broad categories, manual, automatic and automatic segmentation. The analysis of the situation in the area of segmentation enabled a horizontal view of the area of the solution. As part of tins research work, we have implemented a selected approach to obtain a vertical or more in-depth perspective of the segmentation problem and the solution domain.
To further reduce the complexity of the problem, we have limited the types of video sequences that can be processed. We applied the constraints that there is only one moving object in the scene and that the only motion in the video sequence is the motion of the object of interest. We also assume that there is constant illllUnination and no occlusion of the object by parts of the background.
As mentioned, the purpose of implementing a segmentation strategy was firstly to gain a deeper understanding of the segmentation problem and secondly to practically investigate whether a particular scheme could be developed, after a literary review and a theoretical analysis. to a fully automatic winch. video object generation tool. Therefore, this study concluded with an analysis of the implemented strategy and recommendations to advance this research towards realizing the goal of automatic segmentation of video objects.
![Figure 1-2: The Segmentation Dilemma-Picture of a Dalmatian on Shadowed Background [Fors , 2002]](https://thumb-ap.123doks.com/thumbv2/pubpdfnet/10643193.0/15.846.158.713.99.547/figure-segmentation-dilemma-picture-dalmatian-shadowed-background-fors.webp)
Organisation of dissertation
Extracting Semantic Objects
- Concept of semantic video objects
- Semantic video object extraction strategies
- Motion segmentation
- Investigated approach
The motion of a moving object is usually distinct from the motion of the background and other objects. This equation relates the projected pixel motion (dx, dY) to the partial derivatives I(x,y,t) dt dt. P(x I y) ex P(y I x)P(x) = p(u, v, g I x)p(x) (2.4) P(u, v,glx) denotes the conditional probability density of the observation with respect to segmentation and P(x) is the a priori probability distribution of the segmentation process.
P(u, v,glx) denotes the conditional probability density of the observation given the segmentation and P(x) is the a priori probability distribution of the segmentation process. The second term in the numerator is the conditional probability of the motion field given the segmentation and the search frame. The Taylor series is used to describe the luminance of an image around an object as a function of unknown mapping parameters.
An estimate of the frame difference is then expressed as a function of spatial intensity gradients and motion parameters. A second popular morphological tool, namely watersheds, is used in the decision step of the algorithm.
Model Initialisation
- Change detection mask
- Gaussian filtering
- Canny edge detection
- Results
The variables NC and NR refer to the number of rows and the number of columns in the 2D image, respectively. However, this method only provides a rough estimate of the moving object and must be refined by motion analysis. It has fast movement and the object of interest occupies a small part of the total frame.
This is compensated for in the tracking phases of the algorithm which tracks slow motion and fast motion separately. The main shortcoming of the algorithm is in cases where there is no motion between consecutive frames as the threshold detection converges to the noise. The result from this process is a binary, h image of the moving object skeleton and noise regions.
Pavlidis [Pavli, 1980] approach to object thinning is one of the most widespread and accepted approaches. The Euclidean distance transform replaces each object pixel with the pixel's Euclidean distance to the nearest background pixel. As indicated by N, through N4 in the lower matrix, these are the neighbors to the left, above, and along each of the upper diagonals.
The image clearly shows a more complete extraction of the man's leg including smaller components close to the main large component. In the previous section, time-varying or temporal information was used to obtain a model of moving components. All edge map pixels within a distance of 17 of the connected moving components are selected as the moving object.
Sharifi, Fathy and Mahmoudi [Shari, 2002] present an overview of the most commonly used edge detection methods. Then, a 2-D first derivative operator is applied to the smoothed image to highlight regions of the image with high spatial first derivatives. Since the noise signal does not correspond to the actual objects in the image, when compared to the edge map of the original frame, Figure 3-30(d), the resulting model is an empty frame, Figure 3-30(e).
The studied model initialization technique is based on change detection and therefore requires movement of the object between consecutive images to succeed. These two shortcomings of the model initialization phase necessitate the incorporation of model updating into the trackil)g process.
Object tracking
The model of the object is used as a template that is compared to the original frame for a match. OIl} which is the subset of edge pixels from frame k that form the model of the object to track in the next frame, k+ 1. This problem is particularly severe in the case of a rapidly changing object or when there is partial occlusion of the object.
Furthermore, in case of cluttered background, the majority of pixels in the edge image will not touch the object of interest. In the previous section, the theoretical and practical aspects of the Hausdorff tracker were discussed. The above result could be further improved if less of the background was included in the object model.
The bending motion is fast and causes rapid changes in the shape of the object of interest. Before object extraction, it is necessary to first ensure that the contour forms a closed boundary of the object. The algorithm described above yields a set of pixels, Ch, that form the closed outer boundary of the object.
This section presents the results of the object extraction method discussed in the previous section. The fifth and sixth frames show the object's distance from uncovered background objects. The last frame in the sequence shows the failure of the object extraction as the object moves out of the camera's view.
The movement of the object also occurs within a limited range, revealing very little background. These parts of the background should only be removed if there is more knowledge about the object in question. The moving object is then the difference between the actual frame and the recorded background frame.
Extracting the Object
- VOP extraction algorithm
- Closing the contour
- Extracting the object
- Results
Conclusion
- Summary of work
- Review of possible enhancements to current investigated segmentation approach
- Context of research
This research effort adopted this vision by first reviewing the state of the art in video segmentation. The key finding that was made was that the very nature of the segmentation problem is ill-posed and therefore it is difficult to envisage a comprehensive theory of segmentation. The first assumption or constraint is that the only motion in the video sequence is assumed to be the motion of the video object of interest.
The second assumption is that there is no occlusion of the object, either by the background or other objects. The first phase, model initialization, uses both temporal and spatial information to obtain a model of the moving components. The next phase applies Hausdorff object matching to the tracking and updating of the object as it moves and evolves in the video sequence and the third phase is the extraction of the object from each frame of the sequence.
The main contribution of this developmental effort, that of Meier and Ngan, was the implementation of a global adaptive thresholding algorithm in the change detection phase of the model initialization phase. When this component of the algorithm is removed, the output is processed at 10 frames per second. With optimized coding and a modern computer, increasing the framerate by an order of magnitude would be a feasible goal.
Alternatively, the research can be streamlined towards object-based coding, and the focus will then shift towards describing the contour, motion and texture of the extracted object in a coding framework such as MPEG 4. However, in the context of the current segmentation approach, further studies can be carried out in efficient encoding of contour, motion and texture of the extracted object and in motion analysis of the object as it is tracked along its path. However, it would be more practical to extend the scope of the work to segment more objects that can block each other.
From an analysis of the future path, it is clear that realizing the vision of an automatic video object generation tool would likely result in a complex, compute-intensive system. Chalom, E., Bove, M., September Segmentation of an image sequence using multidimensional image attributes", in Proceedings of the IEEE International Conference on Image Processing, ICIP-1996, Lausanne. E A new approach to linear filtering and prediction problems" , Transaction of the ASME Journal of Basic Engineering, 82 (Series D), pp.
