ADVANCED COMPUTER TECHNOLOGY FOR ASSESSMENT AND
asked question and fij are the proposed answers to this question, i is the current number, j is number of answer in the list of proposed answers, and T is time to test. Here i<=N, j<=Na , N is number of records in the collection, and Na is number of proposed answers. To the sake of simplicity we further refer to the experts (teachers) knowledge base as the etalon KB. We assume that answers fij are ordered on the closeness to the correct answer. Then Fi = fi1 is the correct answer to i-th question.
In general the normalized knowledge Z may be defined as:
Ni
f
iZ N
1
1
, (1)
(We note that before the presentation all data undergo to random mixing.) At evaluating, the question is addressed to the student and his answer is compared with the corresponding correct answer.
After testing the initial collection is extended with obtained results, e.g., obtained answers and other relevant information (response time, rating, etc.). After test, the initial collection is extended by the obtained answers, and there contains all necessary information for the estimation of the student's knowledge.
There are two different types of examination. In first type the student must recognize the correct answer from several predefined variants fij (the closed-ended form of the test). In the second type the system allows the student to write his answer freely (the open-ended form of the test). Obviously that in the first case (closed-ended test) the verification problem of the answer can be solved easily. But this problem requires much more efforts in the case of open-ended test. Since the correct answer can be written in various but equivalent forms, the reduction problem of the answer to the unique referenced form has obvious solutions only for the simplest cases. But in general such a problem may turn out quite involved. Here we pay main attention to the closed-ended test case.
3. Educational security
Due problems of information security are so common today, it is not surprising that problems of
"educational security" could be common too (Rove, 2004). The Faculty who has not taught on-line often asks about cheating. Specifically, they ask how do you know the person who is taking the class is the one who signed up? Unless photo IDs are checked and all course work occurs inside of a monitored classroom, faculty really does not know for sure whether the student is who they say they are in the classroom or on-line (MSU, 2006).
In on-line assessments in which we are not sure who is taking the test; students will be under pressure, some students perform unfairly poorly under pressure and this is a good incentive to cheat (Rove, 2004). We have a wide spectrum of documented techniques to commit cheat on on-line assessments: modify a grade in the database (DB), to steal answers for questions, to copy from another student or cheat sheets, impostor or substitute remote students, to search for answers on the Internet, on the messenger or cellular phone, in single words to "commit cheat" to obtain a "better grade" in an online assessment (Hernandez et al. 2006).
For many kinds of student performances, the potential for cheating in an on-line course is not different than the potential for cheating in a classroom based course (MSU, 2006), besides using traditional mechanisms to cheat, students are using new technologies like cellular phones, i-pods, the messenger and the Internet to commit this dishonest practice.
An on-line assessment is composed by assessment items, each assessment item includes the question, available answers and the information required to be processed (IMS GLOBAL Learning Consortium, 2015). A common practice to avoid cheating is the use of a large pool of assessment items to customize assessments. However, even with a large pool, a different danger is that students may be able to log in as the instructor and read the answer key themselves. Most assessment software is protected by short passwords—in Blackboard these can be as few as eight characters, easy to guess with today's systematic "cracker" software (Rowe, 2004).
4. Use of data mining to detect students cheating and suspicious behaviour
Our discussion focus on the student's behavior under the on-line assessment environment, for instance students are more likely to cheat if they observe other students cheating or if they perceive that cheating is allowed. Our proposed model to help organizations to detect and to prevent cheats in on-line assessments. First we analyze different student personalities, stress situations generated by on-line assessments, and the common practices used by students to make cheat to obtain a better grade on these exams. Later we present our DMDC (Data Mining to Detect Cheats) model based on what we call the
summary of best practices; here we analyze the designed database schema to register the student's information. We used Weka (WEKA, 2015) to carry out data mining to find behavior patterns that fits suspect profiles to detect cheats in on-line assessments. "Weka is unique because it's easy to use and understand, and provides a comprehensive environment for testing methods against other existing methods" (FRST, 2006). Finally, we discuss the results obtained by applying of the proposed model and summarize our conclusions.
Figure 1. KDD Schema and Data Mining
5. Use of Data Mining and Biometric Recognition to Detect Students Cheating and Suspicious Behaviour
5.1. Related work
An advanced security measure can be implemented by means of biometric technologies; they may provide added robustness in access control to high security facilities within higher education. As the unit price for biometric devices continues to fall it is possible to employ these to replace the current systems used for workstation and network access (Wasniowski, 2006). These devices are likely to become a standard computer peripheral, built into future workstations.
The use of biometrics in on-line assessments is new and is planned to use as initial step for on-line assessments of migrant workers in the United Kingdom. Immigrants will complete self-assessment tests on-line "wherever possible" before contacting immigration officials, although the on-line self-assessment would only cover the initial stages of an application. Basic questions will be asked, such as the purpose of a visit to the UK, the planned length of stay, qualifications held and work experience already gained. Lower-skilled people from outside the EU (European Union) will have less chance of entering the country to work, said the government (Savvas, 2006).
Now, is evident to us, the educative and technological importance of biometric recognition, but besides discussing the application of these technologies, our intention is to analyze the impact of its applicability in online assessments to support professors day to day activity, and provide an objective assessment of students over a secure environment. In next section we define the problem at hands; on second section we explain the Methodology we used for our experiment. On the third Section, we describe the characteristics of our made at home Online Testing System with Biometric Recognition (OTSBR), its technical requirements, the performance schema, and its implementation, and other relevant issues resulting from the biometric implementation. At the end of this paper, we present our preliminary results, our future work and conclusions.
The main problem on online assessments is to know who's there? (Hernández, Burlak and Lara, 2010). In this paper, we propose the use of biometrics, particularly the use fingerprint recognition on real time to authenticate students into the assessment system, and web cam monitoring during online assessments to deal with the well-known problem of: who is taking the exam? The contribution of this paper is the use of biometrics on online assessments as a new approach for remote identification on real time, we reviewed the literature on different related fields and we realized there are several proposals to deal with this problem, however none documented implementations of such technologies has been tested with flesh and bone students. Some of these proposals include separated point of views of IT, educational professionals or trainers with different perspectives, that problem was solved by constituting a multidisciplinary team work of professors, psychologists, statistics professionals and IT consultants.
5.2. Methodology
Virtual proctoring involves using biometric technology to monitor students at remote locations.
For virtual proctoring, is recommended using a layered approach depending on critical maturity of the test. With high stakes tests, video monitoring and a biometric measure such as iris scanning may be used.
For medium stakes tests, a single biometrics measure may be acceptable (BSU, 2006). Despite most of online assessments are located in the middle of both definitions, we consider the fact of high levels of cheating in remote assessments. In one hand, fingerprint recognition is a single biometric measure, the
cheapest, fastest, most convenient and most reliable way to identify someone. And the tendency, due to scale, easiness and the existing foundation, is that the use of fingerprint will only increase. Cars, cell phones, PDAs, personal computers and dozens of products and devices are using fingerprint recognition more and more (Griaule, 2015). One current trend is to incorporate fingerprint scanners into personal computers, laptops, and mice. In addition, computer networks and large databases can be secured using fingerprint technology. This is a hot topic of discussion since the phenomenon of the Internet and the development of Intra nets has spawned new digital technologies such as E-commerce and online services (ITEDU, 2006). Besides, users are more willing to use fingerprint recognition than iris recognition (AIDC, 2006), they believe is more secure for health. Unfortunately, fingerprint recognition is used to authenticate into systems, but then what? The student is free to use any media to commit cheat, to avoid that situation we considered the possibility to use web cams. Web cams are inexpensive and most of students are used to deal with them, they form part of their common tools to work and chat. Is for sure that some students will reject the possibility to be monitored, and percentages vary from country to country, but is our intention to measure this figure as a part of our research. Based on above exposed, we propose the mixed use of video monitoring, by means of web cams, and fingerprint recognition to provide a secure on-line assessment environment.
For our experiment, we selected a random sample of students (n=102) from the Jose Maria Morelos y Pavon High School, located in Temixco, Morelos, Mexico. We carried out two evaluations, a control evaluation (paper and pencil), and a second evaluation with our online assessment system with biometric recognition.
Tests design. Tests were designed by professors on August 5th and 6th 2007, one of them was implemented for the online assessment using our authoring tool. The tests consisted of 30 questions with similar level of complexity; we evaluated arithmetic, algebra, geometrics, and trigonometric subjects.
Setting up. Computers were prepared with our online assessment client software and biometric devices, network connectivity were tested.
The traditional test. The paper and pencil test was conducted on August 14th. 2007. This test can be consulted at http://www.on-line-surveys.com/uaem/doctorado/
Enrolment. Students were enrolled into the system by taking their left-hand index fingerprint on August 15th 2007. We took care of the students were identified by the system after their enrolment.
The online assessment with biometric recognition test. Was conducted on the Computers Network Laboratory located at the High School facilities from August 16th to August 17th 2007, each computer used in the experiment had attached a Microsoft Fingerprint Reader, a web cam, a broad band connection to our server as well as our proprietary client system. First of all, Students were instructed in how to use the system, we explained them that a web cam was monitoring their activities, later students authenticate by means of their fingerprint into our Server System and the computerized assessment started. The use of calculator and cellular phones was avoided.
The Survey. At the end of the exam we apply a survey to determine students' profile and perceptions about system's operation. We attached this information to demographics and results of test to perform data mining.
Statistical Analysis. Data was processed using descriptive analysis, using relative numbers and percentages using Ccount gnu free software.
5.3. Preliminary Results and Discussion
Considering the number of students enrolled (n=102) on this test with obtain a FAR of 99.99%
and a FRR of 97.09%, only one student could not be recognized despite several trials, although we try enrolled her trying different fingers of her left hand we could not, she has tiny long fingers and the enrolment results were always the same. Her fingerprint template cannot be understood by the system due is confuse, her fingerprints seems like stains. Something related is registered in literature, Asiatic persons has similar problems to be identified by fingerprint readers (Michigan Org, 2015). We faced this problem by providing this student one user and a strong password. Finger print recognition was very well accepted but web cam monitoring was considered as measure that invades privacy. Students identified some ways to hack biometric system like staining the fingerprint reader or moving the web cam to focus a different place. In average, the online test grade was lesser than the traditional test grade, which means technologic platform influenced the student performance.
6. Conclusions
In this paper we have studied Data Mining (DM) as a tool to detect the student cheats in online assessments. Since the increasingly new technologies evolve, students are mastering cheating in online tests. This allows us to define this fact as cyber cheating due to the use of the Internet as the interaction
media. We also studied biometric recognition to solve the question: Who is there? In online assessments, in this sense fingerprint recognition was very well accepted, meanwhile web cam monitoring must be replaced by a non-disturbing device.
Acknowledgments
Part of this work was supported by CONACYT México project 169496.
References
AIDC (2006). Automated Identification and Data Capture Web Site. Retrieved August, 2006, from http://et.wcu.edu/aidc/
Bugai, Y., Burlak, G., Demchenko, A., Kuz’menko, N. (1997). On Application of Computer Technology Testing In Educational Process, In: Conference Role of Universities in the Future Information Society RUFIS’97, Prague.
BSU (2006). Ball State University. Technology assessment reports on The use of Biometrics in education. Retrieved August, 2006, from http://web.bsu.edu/dpmuller/biometrics/
CISCO (2015). Cisco Systems. Retrieved May 15th , 2015, from http://www.cisco.com/
Carnegie (2005). Justice or Just Us?. Retrieved December 28th, 2005 from http://www.carnegiefoundation.org/perspectives/sub.asp?key=245&subkey=577
FRST (2006). Foundation for Research Science and Technology. Retrieved January 12th, 2006, from http://www.frst.govt.nz/publications/tech-reports/downloads/index/Issue_29.pdf
Griaule (2015). Biometrics FAQ. Retrieved May, 19th 2015, from http://www.griaule.com/page/en- us/faq_biometrics
Hernández, J.A., Ochoa, A., Muñoz, J. and Burlak, G. (2006). Detecting cheats in online student assessments using Data Mining. In Dmin06, Las Vegas Nevada, USA.
Hernández, J.A., Burlak, G. and Lara, B. (2010). Design and Implementation of an Advanced Security Remote Assessment System for Universities Using Data Mining. Computación y Sistemas, 13(4), 463-473.
IMS GLOBAL Learning Consortium (2015). IMS Question & Test Interoperability Specification.
Retrieved May 15th, 2015 from http://www.imsglobal.org/question/index.html ITEDU (2006). Biometrics. Retrieved August, 2006, from http://et.wcu.edu/aidc/
McCabe, D. L., y Trevino, L. K. (1996). What we know about cheating in college: Longitudinal trends and recent developments. Change, 28(1), 28-33.
McCabe, D. L., y Trevino, L. K. (1997). Individual and contextual influences on academic dishonesty: A multi-campus investigation. Research in Higher Education, 38(3), 379-396.
Michigan Org (2015). Fingerprint Classification. Retrieved May 19th, 2015, from http://www.reachoutmichigan.org/funexperiments/agesubject/lessons/prints_ext.html
Morris, T. A. (2005). Cheating and plagiarism in the information age. Retrieved December 30th, 2005, from http://terrymorris.net/tohe2004/CPIA_morris.pdf .
MSU Michigan State University (2006). Quizzes and Exams: Cheating on exams and Quizzes. Retrieved
August, 2006, from
http://teachvu.vu.msu.edu/public/pedagogy/assessment/index.php?page_num=3
Rove, N.C. (2004). Cheating in Online Student Assessment: Beyond Plagiarism. Online Journal of Distance Learning Administration, 2(2). Retrieved May 19th, 2015, from:
http://calhoun.nps.edu/bitstream/handle/10945/36015/Rowe_Cheating_in_Online_Student_Assess ment.pdf?sequence=1
Savvas, A. (2006). Online assessments and biometrics used in new immigration system. Computer Weekly Section IT Management Politics and Law, 10-10, 1/7p.
Wasniowski, R. A. (2005). Using Data Fusion for biometric verification. Transactions on Engineering, Computing and Technology, 72-74.
WEKA (2015). Weka 3: Data Mining Software in Java. Retrieved May, 15th, 2015, from http://www.cs.waikato.ac.nz/ml/weka/