Chapter 1. Introduction 1
1.14 Summary of chapters
This thesis is divided into six chapters is as follows:
Chapter 1: Introduction (present chapter)
The first chapter highlights the overview of Design entrance examinations in India, question patterns, types of question that instigates creative responses from students, patterns of creative responses, categories of novelty, understanding and evaluating novelty in mass examinations of Design institutes. State of the art literature review highlighted methodologies, algorithms related to descriptive and image-based content assessment, and evaluation of question formulation. Subsequently, insight is drawn from literature review and research gap, research questions, aim, objectives, expected outcomes, and structure of this thesis is presented.
Chapter 2: Identifying parameters of creative questions and designing a digitized system to support design pedagogues for framing questions
The second chapter reports the study conducted to identify the parameters of questions that have the potential to instigate creative responses from students. A systematic mixed-method technique is reported that have been used to capture the features of creative questions from experts. Further, design to automate the identification of creative questions is illustrated. The design is transformed into computational models by implementing it using various algorithmic techniques. Finally, inter-rater reliability among the model and examiners was measured, and the outcomes were reported to show the subjective agreement among them.
Chapter 3: Identifying parameters to assess novelty of descriptive creative responses and digitizing its evaluation process
The third chapter describes the comprehensive investigation of identifying dimensions or parameters of evaluating novelty from descriptive creative response. This identification process involves studies with human experts possessing expertise in evaluating creative aptitude. The descriptive statistic of experts involved in the study is presented in this chapter.
Further, design of the computational model to automatically evaluate descriptive creative response is illustrated. Subsequently, the model is implemented by using various algorithms.
Finally, the model is validated by comparing the outcome with human-based assessment.
Chapter 4: Identifying parameters to assess novelty of labelled image-based creative responses and digitizing its evaluation process
The fourth chapter highlighted the investigation of identifying the features of evaluating novelty from labelled image-based creative responses. This identification process involves studies with human experts possessing expertise in evaluating creative aptitude. The descriptive statistic of pedagogues involved in the study is illustrated in this chapter. Further, design of the computational model to automatically evaluate labelled image-based creative response is shown. Subsequently, the model is implemented by using various computational procedures. Finally, the model is validated by comparing the outcome with human-based assessment.
Chapter 5: Identifying parameters to assess novelty of annotated image-based creative responses and digitizing its evaluation process
The fifth chapter describes the comprehensive investigation of identifying dimensions or parameters of evaluating novelty from annotated image-based creative response. This identification process involves studies with human experts possessing expertise in evaluating creative aptitude in design education. The descriptive statistic of experts involved in the study is presented in this chapter. Further, design of the computational model to automatically evaluate annotated image-based creative response is illustrated. Subsequently, the model is implemented by using various computational methods. Finally, the model is validated by comparing the outcome with human-based assessment.
Chapter 6: Discussion and conclusion
The sixth chapter presents a comprehensive discussion of the overall summary and key findings of the thesis. The fulfillment of the objectives are mapped with each of the chapters. A detailed description of the validation of each model is presented. Implications of the research are derived from the perspective of design process and human-centred approach. Further, contribution to knowledge-base, methods, design, and design education are highlighted.
Finally, limitations, future scope, and overall conclusion of the thesis are presented.
Chapter 2: Identifying parameters of creative questions and designing a digitized system to support design pedagogues for framing questions
Abstract
Creative question triggers students' creativity and pedagogues attempts to capture it by the creative questioning technique. Creative questions are a major component of examination in Design education for testing creative aptitude. Examiners drill their thought processes and grapple with their ideas to frame questions that are creative in nature, which is capable of capturing creative aptitude in students. While framing creative questions, examiners often self- evaluate, compare, and contrast their ideas before finally phrasing the question. During this process, they remain ever-inquisitive to know whether questions framed by them are really creative; to be more precise, do the questions framed by them really capture features of creative questions? Peer review in these situations provides meaningful insights into construction of these questions. However, peer review has its own demerits. Individual characteristics and past experiences influence frame of reference of all individuals. And as such peer-reviewing a question paper might lead to more differences than convergences. The investigation presented in this thesis is exactly geared towards this issue. Our objective is to explore whether technology can support examiners in situations like these. Nowadays, increasing number of Deep Learning (DL) techniques are widely applied in Question-Answering (QA) platforms to assess success of a question. DL is often used to recognize features of questions. However, it has been hardly used to identify creative features in questions that attempt to trigger creative response from students. The study presented here, investigates features of creative questions through mixed-method research techniques. A model is proposed based on DL algorithms that can find out inherent creativity factors in questions and identify whether a question is creative.
This process of identifying creative questions triggers decision-making of examiners by which they update their questions based on the outcome of the DL-based system. This model is implemented using Bidirectional Encoder Representations using Transformers (BERT) and Long Short Term Memory (LSTM) method for identifying creativity in questions and their performance is compared. Results highlight that BERT overrules LSTM mechanism, thereby optimizing the trust in BERT algorithms. A comparative study between the outcome of the model and examiner’s opinion of categorizing creative questions are mapped, thereby further building trust in the model. A major contribution of this research is to capture creative features in a question and categorize whether a question is creative in design education. This model
highlights human-machine collaboration and promotes examiners' decision-making process to frame effective questions. It attempts to reduce uncertainty of examiners and assists in quick decisions to include creativity features in their questions by providing feedback on whether a question is creative.
Highlights
• Human-centred design approach to identify parameters of questions that has the potential to instigate creative responses from students.
• Proposing a computational design model for assessing creative questions.
• Implementing the model using various tools and algorithmic techniques.
• Validating the model by measuring the inter-rater agreement.