Watermarking of 3D Images and Video

In the first part of this chapter, a watermarking scheme is proposed to secure depth information of the sea. 1.1(c), 1.1(d) shows the dependent and independent views of the left and right views. a) Image in left view (b) Image in right view.

Depth

DIBR-3D Representation

Left and right views are obtained by rendering the center view using the DIBR technique [8]. In DIBR-3D views, the gaps in the independent regions are filled using the gap filling process [9], where the average values of the border pixels are used to fill the regions [1] as shown in Fig.

Multi-view Representation

It is noted that, due to the gap filling technique, the independent areas provide a horizontal line pattern for the left and right views of the DIBR-3D image (see Figure 1.3). The encoder identifies areas in the current image that can be displayed from images of the same temporal instance in a reference view based on the reconstructed depth maps of the reference view.

Digital Watermarking

Efficiency Parameters

Literature Survey

The authors used an object-based watermarking scheme where the watermark was embedded in each frame of the left and right eye video with a selected object from the video sequence to embed the 3D characteristics. Also the watermark in the left and right view is opposite to each other.

Motivation and Objectives

It proposes a robust depth watermarking of 3D image and video sequences to resist various depth-based attacks [76,80]. Maintaining decent visual quality of watermarked images and video sequences according to HVS using appropriate coefficient selection methods.

Contributions of the Thesis

Robust Watermarking for MVD Representation against 3D-

Robust Watermarking for MVD Representation
Watermarking in Image Depth against View Syn-
Watermarking in Video Depth Sequences against

In the first phase of this work, a video watermarking scheme is proposed to resist the 3D-HEVC compression attack, where an independent region (say, Z-axis) is used to insert the watermark. In the second phase, an extended version of the work of the first phase is presented to provide depth to the 3D video sequences in the MVD presentation.

Depth-based View Invariant blind 3D Image Watermarking

The scale-invariant feature transform (SIFT) [81] is used to find the view-invariant feature point locations from the main views and the watermark bits are inserted into the depth of the obtained locations to make the schema invariant for the view synthesis process. A set of experiments is performed to justify the robustness of the proposed scheme as the existing scheme.

View Invariant Watermarking Using DT-CWT for DIBR-

Thesis Organization

Then, another DT-DWT based watermarking scheme is presented that allows MVD to resist both the display synthesis and 3D-HEVC compression attack. A novel watermark embedding scheme is presented that uses the DT-CWT to resist the display synthesis attack.

Summary

In [86], the motion compensation has been performed to generate the center frame from the boundary frames as proposed by Atta et al. To generate the video sequence, DCT-based inverse motion compensated time filtering (IMCDCT-TF) is performed over the time-filtered video sequence .

Semantic Image Segmentation

CRF-based probabilistic graphical modeling for structured prediction is processed with the pixel-level prediction and semantic segmentation task is done by combining the end-to-end trainable deep network by a fully convolutional neural network with the CRF-RNN.

Scale Invariant Feature Transform

Dual Tree Complex Wavelet Transform (DT-CWT)

Furthermore, it is experimentally observed that the magnitude change of DT-CWT coefficients due to DIBR is much smaller. So the value of DT-CWT coefficients will remain constant than DCT of FFT coefficients against DIBR based image synthesis process.

Attacks on 3D Image and Video

Evaluation Parameter

Visual Quality Parameter

PSNR
SSIM
PSNRHVS
MSSSIM
VIFp
Flicker Metric

The VIFpi calculated for a collection of N × M wavelet coefficients from each subband using the following Eq. EN|sN) represents the information that could ideally be extracted by the brain from a certain subband in the reference and test images, respectively. CN,j represents N elements of Random Field (RF) Cj describing the coefficients from sub-bandj [94].

Robustness Parameter

Experimental Dataset

Summary

Proposed Concept

To increase the robustness of the watermarking scheme against video compression [86], motion-compensated Z-axis regions are taken for watermark embedding in a given GOP of a 3D video sequence. The fully connected regions of the Z-axis of the compensated frame for GOP(=8) are depicted in Fig.

Proposed Scheme

Watermark Embedding
Watermark Extraction

After motion compensation, a location map containing the watermark bit and the location of the two corresponding 4×4 blocks is generated for each frame. Like the left eye video, repeat step 1 to step 5 for the right eye video to extract the watermark from the right eye video Wr0.

Experimental Results

Visual Quality Measurement
Robustness Analysis

Proposed left eye (view=2) Lee's schematic left eye (view=2) Proposed right eye (view=4) Lee's schematic right eye (view=4). Proposed left eye (view=2) Proposed right eye (view=4) Lee's scheme left eye (view=2) Lee's scheme right eye (view=4).

Discussion

Watermarking for MVD Representation against 3D-HEVC Com-

Proposed Scheme

Embedding Zone Selection

After embedding, watermarked center view is rendered to get watermarked version of left and right view which is communicated to receiving side. These watermarked center views are rendered inversely to generate the dependent part of left and right views and merged to finally generate watermarked views.

Experiment Results

Visual Quality
Robustness

It can be seen from Table 3.4 that the proposed scheme produces almost similar result for the PSNR, SSIM and VIFpa compared to Lee's scheme [65]. It is noted that the proposed scheme performs better against collusion attacks than Lee's scheme [65] and Rana's scheme [100] because the watermark in the dependent view will create co-located regions for the left and right views.

Discussion

Summary

Proposed Scheme

Detection of Background and Foreground Objects 70

In this scheme, the watermark is embedded in the depth map (image) of the image in the center view. In this scheme, SIFT locations of the original image are used to embed the watermark into the depth image.

Results

Visual Quality
Robustness
Discussion

To compare the visual quality of the 3D image, focus Table 4.1: Average PSNR comparison result, SSIM of the proposed scheme with Guan's scheme [79] for depth. The robustness of the proposed scheme is evaluated against JPEG compression of the depth image.

Watermarking in Video Depth Sequences against Depth Modifica-

Proposed Scheme

Motion Compensated Temporal Filtering (MCTF) 81

After motion compensation, the connected locations are mapped to the first frame of the IDP. MCTF is performed on each IDP of the center view video frame (as described in Section 4.2.1.1) and only the connected regions are detected.

Results

Visual Quality
Robustness
Discussion

The comparison of the proposed scheme with the existing one [79] for different levels of QP is shown in table 4.7. To test the robustness of the proposed scheme against view synthesis, the depth of the video sequence is synthesized at the different position between the left and right views.

Summary

Dependent View Region Identification
Layer Partitioning using Depth
Block Partitioning
Block Selection
Visual Threshold Checking

In this work, identical watermark is embedded in the dependent regions of the left and right viewport. 5.1(a)) is given in the right view to generate the dependent view regions of the right view.

Watermark Embedding and Extraction Process

Embedding of Watermark

A simple embedding rule was used for the proposed scheme as described in Eq. 5.3) where Wi is the ith watermark bit to be embedded and (C1 and C2) are the DC coefficients of the selected block pair. After embedding the watermark by varying the DC coefficient of the blocks, blockwise inverse DCT was done to recover the watermarked left-view image.

Extraction of Watermark

Dependent View Region Identification for Synthe-
Visual Threshold Checking

Similar to step 1, the right-dependent view area Vdr for right IR is generated as described in Figure 5.1. Similar to the embedding process, dependent display region identification (see §5.1.2) was used to find the dependent display regions for left and right views.

Robustness of the Proposed Scheme

In this work, the block partitioning was done row by row for the dependent view area. To handle this situation, in the proposed scheme, pairs of blocks are selected from rows of dependent view regions that have width.

Experimental Results

Visual Quality

The visual quality performance of the proposed scheme against the above metrics for different embedded block sizes (4 × 4, 8 × 8, and 16 × 16) is shown in Table 5.2. Table 5.2 shows that the performances of the proposed scheme in terms of various visual quality metrics such as PSNR, SSIM, VIFp, etc.

Robustness

For both cases, the proposed scheme shows relatively better results than the existing schemes. It is noted that the proposed scheme outperforms the above-mentioned recent existing schemes against stealth attack [75], where the image on the right (camera view 5) is rendered to the image on the left (camera view 1) and paired with .

Discussion

In this stealth attack, the independent view regions of the synthesized left view (generated using DIBR technique) are filled using hole filling technique [9]. d) Average of Middlebury Stereo 2006 dataset of 21 images. This is the intuitive reason why the proposed scheme outperforms other existing schemes when embedding and extraction views are different as shown in Fig. d) Average of Middlebury Stereo 2006 dataset of 21 images.

Summary

Rendering of DIBR-3D-Image

The base distance is used to calculate the offset between the left and right view pixels. To obtain the left and right view, the canter view is rendered using the DIBR technique.

Dual Tree Complex Wavelet Transform (DT-CWT)

Moreover, it has been experimentally observed that the change in the magnitude of the DT-CWT coefficients due to the DIBR is much smaller, as discussed in §2.4. By observing the characteristics of the DT-CWT (as shown in Fig. 6.2), it can be said that the values of H1, H6 are maximum and H3, H4 are minimum.

Proposed Scheme

Zone Selection

Zone Selection for Embedding
Zone Selection for Extraction

Robustness of the Proposed Scheme

In the proposed watermarking scheme, the watermark can be extracted from any of the original or synthesized views. In this matter, the choice of the DT-CWT coefficients plays a major role in imposing the view invariance properties.

Experimental Results

Visual Quality

The results of comparing the visual quality of the proposed scheme with the existing scheme of Lin & Wu [55], Kim's scheme, Kim's scheme* [76] and Franco's scheme [62] are given in the figures 6.7, 6.8 and 6.9. that the proposed scheme produces almost comparable (sometimes better) result for the PSNR,SSIM, VIFpa compared to Lin & Wu's scheme [55], Kim's scheme, Kim's scheme* [76] and Franco's scheme [62] for a similar embedding charge.

Robustness

It is also observed that the proposed scheme for the same embedding payload provides better visual quality than the existing schemes. We compare the robustness of the proposed scheme with existing schemes against the base distance change attack.

Discussion

Moreover, the robust embedding policy makes the proposed scheme outperform other existing schemes for various attacks. d) Average of Middlebury Stereo 2006 dataset. Time Complexity Analysis: In this proposed scheme, the watermark is embedded with the DT-CWT coefficients of the center view.

Summary

Robust Watermarking for MVD Representation against 3D-

The time-filtered Z-axis blockDCT coefficients are used to increase the robustness of the watermarking scheme. To justify the applicability of the proposed scheme in comparison with the recently existing scheme, a series of experiments were performed for different video sequences keeping parameters similar to 3D-HEVC compression.

Watermarking in Depth Information of 3D Image and Video

An extensive set of experiments was conducted to justify the applicability of the proposed scheme against the existing literature against various attacks. The effectiveness of the scheme is experimentally substantiated against the view synthesis process and various image processing attacks.

Left and right video frames with hidden pixels

Dependent and independent regions of synthesized left view of Aloe

Basic encoding structure of 3D-HEVC encoder with inter compo-

Rendering from a left camera position to a right camera position

Watermarking model

Motion Prediction of the Z -axis GOP

Semantic image segmentation

Depth is also used to generate the left and right views of the 3D image. The watermark is embedded in each depth layer of the left or right view.

DTCWT Coefficients

Block diagram of 3D-HEVC codec using 2 view with inter compo-

Left and right z axis extraction using depth value for Balloons

Motion Prediction of the Z -axis GOP

Connected regions of motion filtered Z -axis frame of Balloons video