Chapter 1. Introduction 1
6.6 Novelties in contribution of the present research
Outcomes of this research have significant contribution to the domain of both knowledge-base and design and development of digitized human-centred assessment of creative aptitude in questions and responses in Indian scenarios. The novelty in contribution of this research are as follows.
6.6.1 Contribution to knowledge-base
Reference questionnaire to identify creative questions and evaluate novelty in creative responses in Design entrance examination
The questionnaire is designed for identifying factors of creative questions that triggers creative responses and factors of evaluating novelty in these responses. This questionnaire is validated and is a contribution to the knowledge-base in this field that could be used by future researchers.
Database of dimensions of creative questions and evaluating novelty in creative responses There is a dearth of information related to features or dimensions of creative questions that triggers creative responses and evaluating novelty for multiple patterns of creative responses.
This research work contributes the specific dimensions of assessing creative aptitude that would support education technology researchers in future.
Database of creative questions and their responses
Specifically, there is a dearth of dedicated database of creative questions that triggers creative responses and a repository of creative responses. Questions and responses has been identified at various repositories, but their primary association was not with creativity aptitude assessment. Therefore, multiple datasets were collaborated from various repositories (Bombay, 2021a, 2021b; Chua et al., 2009; Karpathy & Fei-Fei, 2015; McAuley, 2018), public and private institutions. These data could be used by instructional designers and data scientists to further explore possibilities for better training and evaluation.
6.6.2 Contribution toward methods
A significant contribution from the perspective of methods is the proposed architectures to identify creative questions and evaluate novelty in creative responses. The architectures highlight a structured paradigm and represent a step-by-step problem-solving approach.
Therefore, this architecture could act as a scaffold to identify creative questions and evaluate novelty of creative responses for a large scale Design entrance examination. The proposed architecture was validated by comparing its outcome with human experts' score. A negligible
difference indicates that the proposed architecture is consistent and can be trusted by pedagogues as a substitute for the existing manual evaluation process.
The outcome of the research investigation is significant as it contributes by defining a technique (a model) that not only extracts quantitative values from creative responses, but also devises a way to propose a score that objectively evaluates novelty. Apart from this, the research investigation also contributes a set of formal evaluation indicators that can be used during assessment of questions and their responses by Design pedagogues. This research would provide knowledge to data scientists community in the direction of utilizing multiple data resources and effectively use in scientific techniques, methods, and algorithms for a wide spectrum of applications. Another major contribution would be to the student community who aspires to get admission in Design schools through entrance examinations by a consistent evaluation process. Examinations conducted on a large scale might encounter inconsistency in evaluation process, but the application of a standardized automated system like this attempts to reduce errors in evaluation. This may result in increasing the chance of getting the desired student being selected through the examination.
The methods presented in this thesis is robust that is proved by comparing the results with manual methods. Reliability is also measured among the outcomes of the proposed method and human experts. This method adopted systematic data acquisition and reporting of descriptive statistic, feature extraction, proposal of models, and validation of models. This method is proposed to maintain a structured paradigm and scientific replicability. This method would support the designer and research community in understanding the process of capturing features of evaluating creative responses and creative questions associated with mass examination of Design education. The proposed architectures would support education technology researchers and influence their assessment design process.
Creativity is involved and practiced in all domains. This methods in the future could act as a scaffold for identifying the features of creative questions and evaluating their responses for other domains such as engineering, business, etc. However, an appropriate questionnaire needs to be formulated to capture creativity feature for other domains and evaluate their responses.
Analyzing the acquired data would provide features of creative questions and evaluating novelty of responses of domain of interest. In order to automate the process of identifying creative questions and responses of domain of interest, DL models needs to be trained with
similar types of questions and responses. Further, categorized outcomes of DL models could be compared with the categories made by human experts. Higher agreement between the models and humans would indicate the success of experiments for any domain of interest.
6.6.3 Contribution to Design and Design education
The research investigation reported here focuses on addressing challenges faced by the Design educators community and can be directly related as a contribution from the perspective of Design Praxiology as proposed by (Cross, 1999; Gasparski, 1979). The proposals made in this research work possesses an approach that intends to prepare Design education community specifically Design pedagogues to embrace changes in existing ways of framing creative questions and assessing their responses. This study has addressed a Design problem, which is specifically based on addressing human-errors in the process of evaluation and assessment of creative responses.
Stress is often responsible for errors and inconsistencies in evaluation. To address these problems, this thesis suggested the design of an automation process of identifying creative questions and evaluating novelty from descriptive, labelled-image, and annotated image-based pattern of creative responses. This is a significant contribution to Deign education as it would support pedagogues on a large scale evaluation of creative responses and identifying creative questions, thereby reducing their workload and frustrations of a repeated task.
Consistency in assessment is an inherent criterion in any examination process. It supports in awarding impartial scores to students and unbiased identification of creative questions. Design education involves human experts who manually evaluate creative aptitude on a large scale in mass examinations. During this process, pedagogues often conduct subjective evaluation based on their own referential metrics. This leads to inconsistency in an evaluation process.
Moreover, examinations like this require creative questions that attempt in capturing creativity of students. During formulating these type of questions, examiners compare and contrast their ideas in order to frame questions that can capture creativity of students. However, during this process they are often biased by their past experiences. Therefore, computational models presented in this thesis is a contribution to Design education as they would support conducting assessment of creative aptitude and identification of creative questions based on pre-defined feature set. The methods involved in this models would ensure scientific replicability, thereby
confirming consistency in assessment process. The outcomes of this thesis are significant as it would support increasing trust in subjective evaluation in Design education.
This thesis contributes in design process of assessment system as it support in systematically identifying features of evaluating creative aptitude and identifying creative questions that instigate creative responses. The design of the computational models would support in consistent evaluation and identifying creative questions. The overall findings of this thesis would have high potential to support designers and researchers in designing an assessment process for Design education.