Chapter 1. Introduction 1
4.2 Method
4.3.2 Descriptive statistic of features of image with labels identified by
Questionnaire was framed to capture features for evaluating novelty from an image with label pattern of creative responses, as shown in Appendix D. A feature list was provided in questionnaire consisting of following items- 1) relevance, 2) uniqueness, 3) clarity, 4) sketching ability, and 5) choice of colors (Al-Homoud, 2020; Charlet & Damnati, 2017;
Chaudhuri et al., 2020, 2021b; Gagnon et al., 2019; Sangkloy et al., 2017; Sarkar &
Chakrabarti, 2011; Wang et al., 2017; Xueqing et al., 2018). Moreover, expert’s opinion on additional features was also accepted in the survey. Initially, these features were received in a pilot study and later included in feature list of questionnaire. Four features were included namely- 1) process, 2) simplicity, 3) imaginative, and 4) versatility. Process refers to a technique of solving a problem. Simplicity of an image with label pattern of creative responses refers to a representation that is easy to perceive. Imaginative indicates a response illustrating creativity and divergent ideas. Versatility represents a response that is adaptable to newer context. Descriptive statistic of all features is illustrated in Table 4.1.
Table 4.1: Descriptive statistic of features for image with labels Statistics
Releva nce
Unique ness
Clarit y
Sketching Ability
Choiceof Colors
Pro cess
Simpl icity
Imagin ative
Versati lity
N Valid 71 71 71 71 71 71 71 71 71
Missing 0 0 0 0 0 0 0 0 0
Mean 1.00 1.04 4.72 4.75 4.72 4.77 4.65 4.76 4.73
Standard
Deviation 0 0.2 0.56 0.52 0.53 0.45 0.63 0.52 0.53
Median 1.00 1.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00
Mode 1 1 5 5 5 5 5 5 5
Frequency of nine features in questionnaire captured from literature and identified from experts in a pilot study are shown in Table 4.2-4.10. It is represented in terms of very important, slightly more important, important, slightly important, and not at all important. The summarized form of captured features derived from descriptive statistics is illustrated in Figure 4.5. In context of mass examination, most of the subjects selected relevance and uniqueness of a response illustrated in dark green colour. Few subjects marked uniqueness of a response as slightly more important. Other features such as clarity, sketching ability, choice of colours, process, simplicity, imaginative, and versatility were chosen as important and slightly important by a few subjects marked in light green and yellow, respectively; while most of them chosen those as not at all important are marked in sky blue. Therefore, relevance and uniqueness of a response were considered as input to the proposed model to evaluate novelty from labelled images due to their relative higher frequency. The internal consistency of the questionnaire measured by Cronbach’s alpha was found to be 0.702 (Gil-Gómez et al., 2017).
Table 4.2: Frequency of relevance in image with label pattern of creative responses Relevance
Frequency Percent Valid Percent Cumulative Percent
Valid very important 71 100.0 100.0 100.0
Table 4.3: Frequency of uniqueness in image with label pattern of creative responses Uniqueness
Frequency Percent Valid Percent Cumulative Percent Valid
very important 68 95.8 95.8 95.8
slightly more important 3 4.2 4.2 100.0
Total 71 100.0 100.0
Table 4.4: Frequency of clarity in image with label pattern of creative responses Clarity
Frequency Percent Valid Percent Cumulative Percent
Valid
important 4 5.6 5.6 5.6
slightly important 12 16.9 16.9 22.5
not at all important 55 77.5 77.5 100.0
Total 71 100.0 100.0
Table 4.5: Frequency of sketching ability in image with label pattern of creative responses Sketching ability
Frequency Percent Valid Percent Cumulative Percent
Valid
important 3 4.2 4.2 4.2
slightly important 12 16.9 16.9 21.1
not at all important 56 78.9 78.9 100.0
Total 71 100.0 100.0
Table 4.6: Frequency of choice of colours in image with label pattern of creative responses Choice of colours
Frequency Percent Valid Percent Cumulative Percent
Valid
important 3 4.2 4.2 4.2
slightly important 14 19.7 19.7 23.9
not at all important 54 76.1 76.1 100.0
Total 71 100.0 100.0
Table 4.7: Frequency of process in image with label pattern of creative responses Process
Frequency Percent Valid Percent Cumulative Percent
Valid
important 1 1.4 1.4 1.4
slightly important 14 19.7 19.7 21.1
not at all important 56 78.9 78.9 100.0
Total 71 100.0 100.0
Table 4.8: Frequency of simplicity in image with label pattern of creative responses Simplicity
Frequency Percent Valid Percent Cumulative Percent
Valid
important 6 8.5 8.5 8.5
slightly important 13 18.3 18.3 26.8
not at all important 52 73.2 73.2 100.0
Total 71 100.0 100.0
Table 4.9: Frequency of imaginative in image with label pattern of creative responses Imaginative
Frequency Percent Valid Percent Cumulative Percent Valid
important 3 4.2 4.2 4.2
slightly important 11 15.5 15.5 19.7
not at all important 57 80.3 80.3 100.0
Total 71 100.0 100.0
Table 4.10: Frequency of versatility in image with label pattern of creative responses Versatility
Frequency Percent Valid Percent Cumulative Percent
Valid
important 3 4.2 4.2 4.2
slightly important 13 18.3 18.3 22.5
not at all important 55 77.5 77.5 100.0
Total 71 100.0 100.0
Figure 4.5: Summary of preference of features of image with labels 4.3.3 Result of prediction of image with labels
For training the data on VGG-19 model, 20,539 images were considered from NUS-WIDE dataset. They were pre-processed and converted into sketches. The sketches were randomly distributed in a proportion of 80% and 20% for training and validation, respectively. The number of training samples was 16,431 sketches with 81 multi-class labels, and the number of validation samples was 4,108 sketches with 81 multi-class labels. The training accuracy obtained was about 64.28%, with a loss of 0.0773 after 30 epochs by using image augmentation technique to reduce overfitting of model and considering a batch size of 128 images. The testing accuracy of 57.31% was obtained with a loss of 0.1114 after 30 epochs by using image
0 10 20 30 40 50 60 70 80
Not at all important Slightly important Important
Slightly more important Very important
augmentation technique to reduce overfitting of model and considering a batch size of 128 images. The results of few predicted test images are illustrated in Figure 4.6.
Figure 4.6: Prediction of labelled images
The prediction illustrates the images and class labels. However, input to the model was images in labelled form, precisely each part of image of image was marked. Using an online OCR application (Best Free OCR API, Online OCR, Searchable PDF - Fresh 2021 On-Premise OCR Software, 2021), texts were recognized in images. Bounding boxes were created around text in images using inverse masking. Further, these bounding boxes were removed using inpainting technique using pre-defined functions in OpenCV and filled the area within bounding box with the average of neighbouring pixels up to a certain radius (Inpainting — OpenCV 2.4.13.7 Documentation, 2021; OpenCV: Image Inpainting, 2021). However, the threshold of radius was pre-defined and not manipulated in present context. Outcome suggests descent results. The overall steps of this pre-processing is illustrated in Figure 4.7.
Figure 4.7: (a) Image with label responses, (b) image with bounding boxes, and (c) removal of bounding box and filling up with average of neighbouring pixels
4.3.4 Relevance score between question and image with label pattern of creative