• Tidak ada hasil yang ditemukan

Manajemen | Fakultas Ekonomi Universitas Maritim Raja Ali Haji joeb.79.4.217-224

N/A
N/A
Protected

Academic year: 2017

Membagikan "Manajemen | Fakultas Ekonomi Universitas Maritim Raja Ali Haji joeb.79.4.217-224"

Copied!
9
0
0

Teks penuh

(1)

Full Terms & Conditions of access and use can be found at

http://www.tandfonline.com/action/journalInformation?journalCode=vjeb20

Download by: [Universitas Maritim Raja Ali Haji] Date: 12 January 2016, At: 23:24

Journal of Education for Business

ISSN: 0883-2323 (Print) 1940-3356 (Online) Journal homepage: http://www.tandfonline.com/loi/vjeb20

Overconfidence and the Performance of Business

Students on Examinations

Paul Sergius Koku & Anique Ahmed Qureshi

To cite this article: Paul Sergius Koku & Anique Ahmed Qureshi (2004) Overconfidence and the Performance of Business Students on Examinations, Journal of Education for Business, 79:4, 217-224, DOI: 10.3200/JOEB.79.4.217-224

To link to this article: http://dx.doi.org/10.3200/JOEB.79.4.217-224

Published online: 07 Aug 2010.

Submit your article to this journal

Article views: 24

View related articles

(2)

ecause our society emphasizes meritocracy and fairness, a stu-dent’s performance on objective and unbiased examinations has far-reaching consequences. For example, a high school student’s performance on the SAT plays a major role in his or her acceptance at particular universities or colleges. Similarly, a business school student’s performance on the GMAT will help decide which MBA programs will accept him or her. Furthermore, a student’s performance on examinations while in college generally determines the student’s course grades, which col-lectively determine the student’s GPA, which often serves as an important merit criterion.

Because of the heavy reliance on a candidate’s examination performance, a significant amount of research has focused on examination instruments and factors that affect students’ performance. Studies of performance on cognitive examinations have shown that, among other things, a candidate’s performance is influenced by subgroup differences (Bobko, Roth, & Potoksy, 1999), envi-ronmental test bias (Reynolds & Brown, 1984), biological and genetic factors (Jensen, 1985), the location of the cor-rect answer in a choice of answers (Bar-Hillel & Attali, 2002), and the test taker’s confidence (Koriat, Lichtenstein, & Fischhoff, 1980).

The effect of students’ confidence on test performance, especially on multiple-choice examinations, is of particular interest to us because most professors who rely on frequent use of multiple-choice examinations to test students have encountered a fair share of overconfident students. After taking a multiple-choice examination, such students often declare that it was easy and that they would either “ace” it or do well, only for the professor to discover that those confident students did not even pass the

examina-tion. In other words, their declaration was not in accord with their perfor-mance. Are such student declarations empty bravado designed to impress the professor or a product of genuine mis-taken assessment by the student? Although it is possible that some of those declarations are meant to impress, it is equally possible that some of them, if not most, are a result of overconfidence.

Although this is an intriguing phe-nomenon that has direct implications on how students and thoughts are evaluat-ed, to the best of our knowledge no study to date has investigated the effect of confidence on the performance of business students on examinations. Given the large differences among dis-ciplines and the fact that different disci-plines attract different personality types, it is possible that students in one disci-pline do not behave as those in another discipline. Thus, in this study we sought to investigate the role of confidence in the performance of business students on examinations—specifically multiple-choice examinations.

Multiple-choice examinations are a common means of evaluating the knowledge base of business students. An investigation of how well the testing method assesses students’ true knowl-edge would be useful for several rea-sons. First, it will provide professors

Overconfidence and the

Performance of Business Students

on Examinations

PAUL SERGIUS KOKU ANIQUE AHMED QURESHI

Florida Atlantic University University of Tampa Fort Lauderdale, Florida Tampa, Florida

B

ABSTRACT. In this study, the authors investigated the role of over-confidence on students’ performance on multiple-choice examinations. The authors examined the difference between students’ self-assessed proba-bility of selecting each correct answer and the proportion of correct answers that they actually selected. Results show that high-performing students discriminated better between difficult and easy multiple-choice questions than did low-performing students. Poorer students exhibited significant overconfidence with difficult ques-tions. Generally, students displayed underconfidence with easy questions and performed better on multiple-choice questions when they were asked to provide reasons contradicting their choices. These results suggest that students can be trained to perform better on standard multiple-choice exams through preparation involving their providing reasons contradicting their choice of answers.

(3)

with an insight into a popular evaluative instrument. Second, understanding the reasons for students’ performance on these tests will be useful in shaping the ways in which instructors teach and evaluate.

Overconfidence, defined as the unwar-ranted belief in the accuracy of one’s judgments (Lichtenstein, Fischhoff, & Phillips, 1982), has been the subject of several studies in psychology. Overcon-fidence could result from the tendency of an individual to seek confirmatory evidence while ignoring contradictory information. We argue that confidence, or lack of it, is relevant to performance on examinations, particularly multiple-choice examinations, because an over-confident student, according to the the-ory that individuals seek confirmatthe-ory evidence while searching memory to retrieve information (Arkes, Dawes, & Christensen, 1986; Hoch, 1985; Koriat et al., 1980), may feel that he or she already knows the correct answer. Because of this “feeling,” the individual may not see the need to search thor-oughly among the given alternatives for the less obvious but correct answer. Consequently, this tendency will lead to poor performance.

Studies in psychology on judgment and overconfidence have shown that people tend to be overconfident when estimating the probability that their judgments are correct (Lichtenstein et al., 1982). For example, the average estimate provided by individuals who were asked to estimate the probability that they selected the correct answers on a multiple-choice examination was found to be consistently greater than the percentage of questions that they answered correctly. Very often, an over-confident individual who estimates that his or her answers to multiple-choice questions have a 90% chance of being correct will find that far less than 90% of the answers are actually correct (Lichtenstein et al., 1982). However, if the individual is well calibrated, his or her confidence would be in accord with the level of actual performance.

If overconfidence were simply the result of mistranslating an individual’s subjective beliefs into numbers, one would expect that individuals would show underconfidence as well as

over-confidence. However, previous studies have shown that individuals are general-ly overconfident rather than underconfi-dent (Koriat et al., 1980; Lichtenstein et al., 1982; Lichtenstein & Fischhoff, 1977; Sniezek, Paese, & Switzer, 1990; Trafimow & Sniezek, 1994; Wright, 1982). Although this issue is interesting, the question of why individuals tend to err mostly on the side of overconfidence falls within the domain of psychology and outside the scope of this study.

It has been argued that the basis of overconfidence lies in an individual’s tendency or bias to seek confirmatory evidence while searching memory to retrieve information. Studies on the relationship between overconfidence and performance on multiple-choice questions have shown that performance correlates highly with the extent to which the test taker searched for evi-dence when considering alternatives (Koriat et al., 1980). Individuals who fully considered each alternative among a given set of choices were better cali-brated than individuals who tended to seek confirming evidence for one alter-native. The implication of this finding is that students who show overconfidence may be biasing their search toward a tentatively preferred answer and thus may not be evaluating fully other alter-natives. Thus, it is possible to improve students’ performance on multiple-choice examinations by making them develop better calibration through the required careful consideration of all alternative answers.

Literature Review

Previous studies have shown that overconfidence could be caused in three different ways. First, overconfidence could result from mistranslating one’s subjective beliefs into numbers. Second, overconfidence could be exhibited dur-ing the postdecisional stage. Durdur-ing this stage, an individual who is in the process of making an overt choice increases his or her personal commit-ment to a selected answer. Third, over-confidence could be exhibited during memory search and retrieval stage. Dur-ing this process, the individual initially selects a tentatively preferred answer. Thereafter, the individual slowly biases

his or her search by focusing on evi-dence that tends to confirm that initial selection. The tendency to seek confir-matory evidence increases the individ-ual’s confidence in his or her answers. These three processes, however, need not be mutually exclusive (see Koriat et al., 1980; Sniezek et al., 1990).

We argue that if overconfidence is simply a matter of mistranslating an individual’s subjective beliefs into num-bers, then one must expect individuals to exhibit underconfidence as well as overconfidence. However, the predomi-nant finding of the previous studies is that individuals generally exhibit over-confidence and not underover-confidence.

This absence of underconfidence sug-gests that miscalibration could not be solely a result of the fact that individu-als simply mistranslate their subjective beliefs into numbers. Furthermore, pre-vious studies also have shown that over-confidence could not be the result of an individual’s personal commitment to a chosen alternative during the postdeci-sional stage. Instead, overconfidence was found to result from the tendency to seek confirmatory evidence in the mem-ory search and retrieval stage (Sniezek et al., 1990). If that is the case, then it would seem that the tendency to be overconfident could be reduced.

Previous attempts at reducing over-confidence have had only marginal success. Arkes, Christensen, Lai, and Blumer (1987), in one of the early attempts to reduce overconfidence, conducted two experiments. In the first experiment, the researchers provided the subjects with feedback on their performance. The subjects in the sec-ond experiment were informed that they would have to justify their answers in a group discussion after they answered the questions. Both techniques were somewhat successful in reducing overconfidence.

However, in an earlier study, Koriat et al. (1980) achieved reasonable success in reducing overconfidence by asking indi-viduals to list reasons why their answers might be incorrect. The results from that study were particularly insightful and indicated that people tend to be overcon-fident because they generally focus on positive rather than negative evidence. The overconfident individual seemed to

(4)

consider reasons supporting rather than contradicting his or her tentatively pre-ferred answer. Because overconfidence results from concentrating on only the positive, an effective way to force an individual to consider other alternatives carefully is to require him or her to pro-vide possible reasons why his or her chosen answer could be wrong.

In this study, we extend previous studies in several ways. Because the focus of previous studies was on docu-menting the existence of overconfidence and analyzing its underlying processes, those studies did not analyze specifical-ly the effect of overconfidence on per-formance. Furthermore, because previ-ous researchers generally used very difficult general knowledge questions to investigate overconfidence, these stud-ies provided little or no room for improvement in performance through deliberation. In such situations, the indi-viduals either knew the answer or they did not. The role of deductive reasoning and analysis of such questions was thus limited.

Our study is different from previous studies in that our orientation is toward application of current understanding of overconfidence. We eliminated the limi-tation of previous studies by evaluating overconfidence through (a) course-specific materials, (b) the use of ques-tions whose levels of difficulty were pre-determined by the authors of the text-book that we used (i.e., the questions were taken from a test bank that was pre-pared by the authors of the textbook and published by the publishers of the text-book), and (c) relating overconfidence to the performance of business students on multiple-choice examinations. This study also extends Qureshi (1998) and Koku and Qureshi (1999) by including business students in general, regardless of declared major.

Hypotheses

On the basis of previous studies (Arkes et al., 1987; Koriat et al., 1980; Lichtenstein & Fischhoff, 1977), we hypothesized that a student’s confidence and judgment on a multiple-choice examination are influenced by the diffi-culty level of the examination questions, the student’s performance, and the

stu-dent’s reasoning mode(i.e., whether the student provided reasons contradicting or supporting the multiple-choice answers). Using previous studies for guidance, we proposed several hypothe-ses for testing the effect of overconfi-dence of students on their performance on multiple-choice examinations.

We formulated our first hypothesis (H1) to test the effect of question diffi-culty on students’ overconfidence. Gen-erally, students who know their material well should be able to discriminate between easy and difficult questions. All other things being equal, one would expect to see an indirect relationship between question difficulty and confi-dence; that is, students who are very familiar with the subject matter should exhibit greater confidence with easy questions and less confidence with diffi-cult questions. However, extrapolations from previous studies lead to a counter-intuitive result. Given what is known from previous studies, one would expect that students will tend to exhibit over-confidence with difficult questions rather than with easy questions. Why? This will happen because, when taking multiple-choice examinations, most stu-dents tend to either underestimate the level of difficulty of difficult questions or fail to appreciate fully the true impor-tance of difficult questions. Thus, we formulated our first hypothesis:

H1: Overconfidence will increase as the level of difficulty of questions increases.

Through the second hypothesis (H2), we considered the effect of student per-formance on confidence. Because the better students (high performing) are better calibrated than poorer (low per-forming) students, the former should discriminate better between what they know and what they do not know. Thus, the high performing students will exhib-it lower overconfidence wexhib-ith difficult questions than would low performing students. In plain language, overconfi-dence seems to result from a false sense of “security,” which poorer students may indulge in. Because low perform-ing students are poorly calibrated, they would be less discriminating and, unlike high performing students, seek confir-matory evidence that falsely increases

their confidence. Thus, we formulated our second hypothesis:

H2: Overconfidence will increase as the student’s performance decreases.

Through our third and fourth hypotheses (H3 and H4), we examined how overconfidence could be reduced. Specifically, we examined the effect of asking students to provide reasons that contradict or support their choice of answers.

On the basis of Koriat, et al. (1980), we argue that students realize the limi-tations of their knowledge when they are placed in a situation that forces them to reexamine their choices critically. Such an environment could be created when students are required to provide possible reasons why their answers could be wrong, along with their chosen answers. By critically reexamining their choices, the students would realize that a question that they initially thought was easy was not easy after all. We for-mulated our third hypothesis:

H3: Overconfidence decreases when students are required to provide reasons contradicting their multiple-choice answers.

Furthermore, because requiring stu-dents to provide possible contradictory reasons why their answers could be wrong forces them to appreciate the full import of a question, they consequently would develop a realistic level of appre-ciation for questions that they initially thought were easy. On the other hand, asking students to provide reasons sup-porting the choice of a multiple-choice answer allows them to escape from carefully examining evidence that con-tradicts the choice. As such, they will become overconfident with their choice of answers whether these answers are correct or not. Thus, we came up with our fourth hypothesis:

H4: Overconfidence increases when students are required to provide reasons supporting their multiple-choice answers.

We formulated our fifth and sixth hypotheses (H5 and H6) to examine whether students’ performance on mul-tiple-choice examinations could be enhanced through a test format. Specifically, these hypotheses test the

(5)

relationship between students’ perfor-mance and asking students to provide reasons that support, or reasons that contradict, their choice of answers.

One would expect that performance would improve as overconfidence decreases, because a decrease in confi-dence with regard to how a student has performed on an examination suggests that the student appreciates the “true” difficulty of questions and does not have a false sense of security regarding his or her performance. If requiring students to provide reasons contradicting their choice of answers to multiple-choice questions reduces overconfidence (as stated in H3), then performance can be expected to improve as the student con-siders all the available evidence.

When forced to seriously consider possible contradictory evidence, a stu-dent is less likely to err by selecting the initially preferred, inaccurate answer; hence performance will improve. On the other hand, asking students to provide reasons supporting their choice of answers to multiple-choice questions will increase the students’ overconfi-dence (as stated in H4). Because the stu-dent is not likely to consider all the available evidence, the student’s perfor-mance will suffer. We therefore formu-lated the following hypotheses:

H5: Students’ performance on multiple-choice examinations will improve when they are required to pro-vide possible reasons that contradict their chosen answers.

H6: Students’ performance on multiple-choice examinations will suf-fer when they are required to provide possible reasons that support their cho-sen answers.

Method

We collected data from 91 university undergraduate seniors and juniors major-ing in business. These students voluntar-ily participated in the study; however, all those who participated in the study were given five extra points toward their course grade. We used the final examina-tion in a capstone business course taken by all business majors, regardless of major, as an instrument for the study.

The examination questions were

taken from a test bank that was written by the authors of the textbook used in the course and published by the text-book publishers. The examination con-sisted of 50 multiple-choice questions, and each question had five possible answers. These questions already were categorized into three levels of difficul-ty (easy, intermediate, and difficult) by the authors of the textbook, who also wrote the test bank. The students were given a maximum of 2 hours to take the examination. Everyone finished the examination during the time provided.

Because the examination questions were based on a textbook that was cov-ered completely by us—the authors of this study—during the course of the semester, we did not conduct any inde-pendent validity and reliability studies on the test questions. However, we both crosschecked questions on the examina-tion to ensure that they were under-standable and dealt with topics that were covered in the course.

Students were randomly assigned to one of the three treatment groups. Stu-dents in treatment group 1 were asked to provide reasons supporting their answers (the “supporting reasons” group). Students in treatment group 2 were asked to provide reasons contra-dicting their choices (the “contracontra-dicting reasons” group), and students in treat-ment group 3 were asked to provide no reason for their answers (the “no rea-son” group). The students in group 1 were asked to write down at least one strong reason supporting their choice of answers for each of the multiple-choice questions. The students in group 2 were asked to write down at least one strong reason why each answer could be wrong. The students in group 3 served as the control group and were asked to not write down any reason for their answers.

We also asked students to write down their subjective probability assessment of selecting the correct chosen answers. In other words, each student was required to indicate by each question how confident he or she was that the chosen answer was cor-rect. The students were allowed to set their lowest probability estimate at 20% and increase this estimate by 20% intervals (because there were five

pos-sible choices) until they reached a maximum of 100%.

We operationalized overconfidence by evaluating the students’ subjective proba-bility assessment of getting each answer right. We computed overconfidence and underconfidence to be equal to the differ-ence between the mean of the probabili-ty responses and the overall proportion of correct answers. This methodology is consistent with Koriat et al. (1980). A positive difference would indicate over-confidence and a negative difference underconfidence. For example, if the mean confidence judgment was 90%, and the mean score of the actual perfor-mance on the examination was 80%, then the subject would be overconfident by 10%. Similarly, if the mean confi-dence judgment was 80%, and the mean score of the actual performance on the examination was 90%, then the subject would be underconfident by 10%.

We used the students’ scores on the final examination to measure their actu-al performance. We preferred using this measure of performance over the stu-dent’s GPA for two reasons. First, the GPA, as a composite of students’ per-formance in all the courses that they took, is amenable to several confound-ing variables because students take courses from several disciplines other than business. For example, in most uni-versities, students are required to take courses that include general studies, sci-ences, mathematics, or the humanities. Second, the method of testing and eval-uating students in other courses, even in the same discipline, could differ signifi-cantly depending on the professor and could range from a take-home examina-tion to merely a term paper. Because of these variables, the GPA could not be a reliable measure of actual performance for this study.

The students’ scores on the examina-tion that we gave ranged from a low of 30 to a high of 98, with a mean of 66 and standard deviation of 13. We classified students in the top third of the actual scores as high performing students. Those students scored 74 or higher. Sim-ilarly, we classified students in the bot-tom third of the actual scores as low per-forming students; they scored 60 or lower. The remaining students were clas-sified as average performing students.

(6)

Results

We summarize the results of our analyses in Tables 1 through 4. In Table 1, we summarize the results of all six hypotheses. The data in Table 2 are a summary of the calculated means for the confidence scores, percentage of correct answers, and over/underconfi-dence for each of the experimental con-ditions. Table 3 is a summary of the cal-culated averages of overconfidence/ underconfidence scores for each group analyzed through ANOVA, and Table 4 is the summary of our ANOVA analysis

of the performance data for students who provided reasons supporting or contradicting their answers and for those who did not provide any reasons.

On the whole, our analyses show that the main effects for performance and question difficulty were significant at the p < .001 level (see Table 3). They also show that the students generally tended to underestimate the difficulty level of difficult questions; thus they became “falsely” overconfident with regard to their performance. In other words, because they failed to appreciate the true difficulty level of difficult exam

questions, they tended to think that they would do better on these questions than they actually did. As a consequence, students’ overconfidence increased with the difficulty level of examination ques-tions. Similarly, because students dis-criminated less between answers when they falsely believed that they already “knew” the correct answer, overconfi-dence increased as the students’ perfor-mance decreased. H1 and H2 were sup-ported by the results.

Our hypotheses regarding (a) the rela-tionship between overconfidence and giving reasons contradicting chosen answers and (b) the relationship between overconfidence and giving reasons sup-porting chosen answers (H3 and H4) were marginally supported by the results (p< .08; see Table 3). Our results show that critical analyses of questions appar-ently helped students appreciate the true import of questions. Thus, students could realize why their chosen answers might be either wrong or correct, if the examination is structured in a way that forces them to think critically about the answers that they proffer.

The results reported in Table 2 are classified into three categories: question difficulty, performance, and reasoning mode. The mean confidence scores on question difficulty show that students were more confident with difficult ques-tions than they were with easy ones. For example, the mean confidence score, which is the mean of the students’ sub-jective probability of selecting a correct response, on easy questions was 84%; however, the actual percentage of cor-rect responses to the easy questions was 86%. Thus, the students were actually slightly (2%) underconfident with easy questions. This is an unexpected finding and suggests that students were uncer-tain when they encountered easy ques-tions and perhaps read more into them than necessary.

The mean confidence score on diffi-cult questions was 73%, whereas the actual percentage of correct responses to the difficult questions was 51%. Thus, the students were grossly (31%) overconfident with regard to difficult questions. For questions with an inter-mediate level of difficulty, the mean confidence score was 77% and the actu-al percentage of correct responses was TABLE 1. Summarized Results From Testing the Six Hypotheses

Hypothesis Findings Sig. of F

H1: Overconfidence increases with the difficulty level

of exam question. Supported 0.001

H2: Overconfidence increases as the students’

performance decreases. Supported 0.001 H3: Overconfidence decreases when students are

required to provide reasons contradicting their Marginally 0.08 multiple-choice answers. supported

H4: Overconfidence increases when students are

required to provide reasons supporting their Marginally 0.08 multiple choice answers. supported

H5: Exam performance decreases when students are required to provide reasons contradicting their

multiple-choice answers. Supported 0.001 H6: Exam performance decreases when students are

required to provide reasons supporting their

multiple-choice answers. Not supported

TABLE 2. Calculated Means for Confidence Scores, Percentage of Correct Answers, and Over- or Underconfidence for Each of the Three Conditions

Over-/under-Confidence Percentage confidence Condition scores (%) correct (%) (%)

Question difficulty

Easy 84 86 –2

Intermediate 77 70 7

Difficult 73 37 36

Performance

Low performing 73 51 22

Average performing 78 68 10

High performing 83 80 3

Reasoning mode

Contradictory reasons 77 72 5

Supporting reasons 80 62 18

No reasons (control) 76 65 11

(7)

70%, indicating a moderate (7%) level of overconfidence.

With respect to performance, the mean probability assessment of high performing students was 83%, and their percentage of correct responses was 80%, indicating a slight level of over-confidence (3%). The mean probability assessment of low performing students was 73%, whereas their percentage of correct responses was 51%, indicating an extreme (22%) level of overconfi-dence. The average-level-performing students had a mean probability assess-ment score of 78%, and their actual per-formance was 68%. Thus, they were moderately overconfident (10%) with their answers.

The results of the “reasoning mode” of the experiment are interesting and in accord with our hypothesis. They show that overconfidence decreased when students were asked to provide reasons why their chosen answers could be wrong. The mean confidence score of the “contradictory reasons” group was 77%, whereas its percentage of correct answers was 72%, indicating only a 5% level of overconfidence. On the other

hand, the mean confidence score of the “supporting reasons” group was 80%, whereas its percentage of correct answers was 62%. This result indicates an overconfidence level of 18% and underscores our argument that students are less discriminating and therefore more likely to select wrong answers when they seek evidence to confirm their initially chosen answers instead of carefully analyzing all the possible answers before making a selection.

The mean confidence score of the control group—the group that did not provide any reasons for or against their choice of answers—was 76%, and its percentage of correct answers was 65%. This result produces an overconfidence score of 11%, a fairly significant amount of overconfidence.

We next investigated whether there was a significant difference among the overconfidence levels of the three groups. To answer this question, we conducted the Bonferroni multiple com-parison test to analyze the differences among the means of the three groups. The results suggest that there was a sig-nificant difference ( p < .05) among the

three groups. However, there was no significant difference (p < .8) between the “supporting reasons” and the “no reasons” groups.

It is evident from these results that asking students to provide reasons that contradict their multiple-choice answers reduces overconfidence. However, ask-ing students to provide reasons that sup-port their multiple-choice answers does not increase their overconfidence. On the other hand, the overconfidence scores were about the same for those who provided supporting reasons and for those who provided no reason. We note that the supporting and the contra-dicting reasons provided by students varied widely in quality; however, an exploratory analysis of the reasons revealed no meaningful relationship.

In Table 4, we provide one-way ANOVA results of our analysis of (a) the relationship between students’ per-formance and asking them to provide reasons that support their choice of answers and (b) the relationship between performance and asking stu-dents to provide reasons that contradict their choice of answers (H5 and H6). Because .0069 is less that .05, the null hypothesis of equal means of perfor-mance level is rejected. In other words, there was a statistically significant dif-ference in students’ performance levels between the group asked to provide rea-sons in support of its choices and the group asked to contradict its choices.

The results reported in Tables 2 and 4 support our hypothesis that asking stu-dents to provide reasons supporting their chosen answers causes them to miss the chance to confront the possibil-ity that they could be selecting the wrong answer. Asking them to provide reasons that contradict their answers not only forces them to think critically about their initial choices, but it also gives them a second opportunity to change their initially chosen answers if the reason(s) for their initial selections are found to be nonconvincing or less logical.

Discussion

The findings of this study represent the preliminary results of our attempt to understand the effect of overconfidence TABLE 3. ANOVA Analysis of Calculated Averages of Over- and

Underconfidence Scores for Each Group

Source of variations SS DF MS F Sig. of F

Between subjects

P (performance) 7307.3 2 3653.75 10.5 0.001 R (reasoning mode) 1815.61 2 907.8 2.61 0.08

P by R 568.39 4 142.1 0.41 0.802

P by R by S (subject) 28533.3 82 347.97 Within subject variations

Q (question difficulty) 55989.85 2 27994.93 157.58 0.001

P by Q 482.73 4 120.68 0.68 0.607

R by Q 1142 4 281.11 1.58 0.181

P by R by Q 914.32 8 114.29 0.64 0.74 P by R by Q by S 29136.38 164 177.66

TABLE 4. ANOVA Analysis of Calculated Averages of Over- and Underconfidence Scores for Each Group

SS DF MS F Sig. of F

Between groups 1711.66 2 855.83 5.2758 0.0069 Within groups 14275.2 88 162.219

Total 159986.9 90

(8)

of business students on their perfor-mance on multiple-choice examinations. Even though we used the multiple-choice format in this study, we believe that the results could be generalized to all types of examinations. We attempt-ed to rattempt-educe overconfidence and improve students’ performance on multiple-choice examinations by asking them to provide reasons contradicting or supporting their chosen answers.

The results provide strong evidence that high performing students appropri-ately exhibit more confidence in their answers. High performing students are also better calibrated and exhibit lower levels of overconfidence. These students are able to better discriminate between what they know and what they do not know. In contrast, although low per-forming students appropriately exhibit lower confidence in their answers, they are poorly calibrated. This suggests that although high performing students are more likely to recognize the extent and limitation of their knowledge, low per-forming students have only limited insight into their performance.

In general, students appropriately exhibit greater confidence with easy questions and lower confidence with difficult questions. However, the stu-dents in this study seemed to be gener-ally poorly calibrated with regard to dif-ficult questions, with which they demonstrated a significant level of over-confidence. This tendency could indi-cate that, on the whole, students do not seem to appreciate the difficulty level of the examination questions. One unex-pected finding of the study is that the students actually exhibited a slight underconfidence with easy questions, which suggests that they were uncertain when encountering easy questions. Thus, they may have attempted to read more into the (easy) questions than was actually warranted or necessary. Per-haps they thought that the easy ques-tions were “trick” quesques-tions.

The results also show that asking students to provide reasons that contra-dict their multiple-choice answers is successful in reducing overconfidence and improving student performance on multiple-choice examinations. This strategy seems to force them to study more carefully the alternative answers

that they had initially rejected. Further-more, it would seem that by concentrat-ing on the weakness of their answers, the students overtly recognize the limi-tations of their abilities, which results in their becoming better calibrated and thus exhibiting lower overconfidence.

In contrast, the results further show that asking students to provide reasons supporting their multiple-choice answers did slightly increase overconfi-dence. However, we observed no statis-tically significant difference between the group that provided reasons support-ing their answers and the group that pro-vided no reasons. This finding suggests that students were already focusing on reasons supporting their answers; as such, asking them to provide supporting reasons did not have a significant impact.

Consistent with previous research in psychology, we found that business stu-dents were overconfident when estimat-ing the probability that their answers to multiple-choice questions would be cor-rect. The results of this study indicate that instructors can remedy this false sense of security that students demon-strate on multiple-choice examinations by changing the format of multiple-choice examinations to require that the student must not only provide the cor-rect answer but also indicate why the correct answer is correct or the incorrect answer is incorrect. We contend that this format not only will improve students’ performance but also will contribute indirectly to learning, as it will help make the student better calibrated and thus less overconfident. A student who is not overconfident will more likely appreciate his or her lack of depth on the subject matter and seek to remedy the situation by studying.

Other equally interesting areas of future research that could influence the way that we evaluate students should include the effect of question format and presentation order on students’ con-fidence and performance on examina-tions. It would be interesting to see if the order of presenting questions in terms of difficulty (e.g., easy questions followed by difficult ones) has any sig-nificant impact on students’ confidence and thus their performance. An argu-ment could be made that if students

encounter easy questions first, they might become overconfident and their exam performance might suffer. On the other hand, if examination questions are ordered from difficult to easy, students might “panic” and the resulting perfor-mance might suffer also. Future research on this issue could provide invaluable insight for professors on how best to evaluate students.

Furthermore, future research on other methods of reducing overconfidence, such as using computerized testing methods that provide students with immediate feedback on their perfor-mance on one question before they move on to the next, might be interest-ing and worthwhile. Future research could also focus on the effect of time constraints on confidence, overconfi-dence, and performance.

The results of this study could have significant implications, given the wide-spread use of the multiple-test format in business education. The results indicate that students indeed can be trained to perform better on the standard multiple-choice exams—such as the Certified Public Accountant (CPA) and Certified Financial Analyst (CFA) exams—when they are prepared through multiple-choice exams that ask them to provide reasons contradicting their answer selections.

REFERENCES

Arkes, H. R., Dawes, R. M., & Christensen, C. (1986). Factors influencing the use of decision rule in a probabilistic task. Organizational Behavior and Human Decision Processes, 37, 93–110.

Arkes, H. R., Christensen, C., Lai, C., & Blumer, C. (1987). Two methods of reducing overconfi-dence. Organizational Behavior and Human Decision Processes, 39,133–144.

Bar-Hillel, M., & Attali, Y. (2002). “The delicate art of key balancing.” Or: When randomization is too important to be trusted to chance. The American Statistician, 564,299–305. Bobko, P., Roth, P. L., & Potosky, D. (1999).

Derivation and implications of a meta-analytic matrix incorporating cognitive ability, alterna-tive predictors, and job performance. Personnel Psychology, 52,561–589.

Hoch, S. J. (1985). Counterfactual reasoning and accuracy in predicting personal events. Journal of Experimental Psychology: Learning, Memo-ry, and Cognition, 11,719–731.

Jensen, A. R. (1985). The nature of Black-White difference on various psychometric tests: Spearman’s hypothesis. Behavioral and Brain Sciences, 8,193–263.

Koku, P. S., & Qureshi, A. A. (1999). Does confi-dence impact the performance on multiple-choice examinations? The case of marketing

(9)

students. Academy of Marketing Science Con-gress, Malta.

Koriat, A., Lichtenstein, S., & Fischhoff, B. (1980). Reasons for confidence. Journal of Experimental Psychology: Human Learning and Memory, 6,107–118.

Lichtenstein, S., & Fischhoff, B. (1977). Do those who know more also know more about how much they know? Organizational Behavior and Human Decision Processes, 20, 159–183. Lichtenstein, S., Fischhoff, B., & Phillips, L. D.

(1982). Calibration of probabilities: The state of the art to 1980. In D. Kahneman, P. Slovic, & A. Tversky (Eds.),Judgment under uncertain-ty: Heuristics and biases. Cambridge: Cam-bridge University Press.

Qureshi, A. A. (1998). Preliminary evidence about the effect of confidence on exam performance. American Academy of Accounting and Finance, December.

Reynolds, C., & Brown, R. (1984). Perspectives on bias in mental testing. New York: Plenum.

Sniezek, J., Paese, P., & Switzer, F., III. (1990). The effect of choosing on confidence in choice. Organizational Behavior and Human Decision Processes, 46,264–282.

Trafimow, D., & Sniezek, J. (1994). Perceived performing and its effect on confidence. Orga-nizational Behavior and Human Decision Processes, 57,290–302.

Wright, G. (1982). Changes in realism of proba-bility assessments as a function of question type. Acta Psychologica, 52,165–174.

Gambar

TABLE 2. Calculated Means for Confidence Scores, Percentage of CorrectAnswers, and Over- or Underconfidence for Each of the Three Conditions

Referensi

Dokumen terkait

Berdasarkan Surat Penetapan Pemenang Nomor : 10/ULP/BPMPD/LS-DS/2012 tanggal 5 Juni 2012, dengan ini kami Pokja Konstruksi pada Badan Pemberdayaan Masyarakat dan

48/VII Pelawan II pada Dinas Pendidikan Kabupaten Sarolangun Tahun Anggaran 2012 , dengan ini diumumkan bahwa

Mengingat sebuah organisasi nirlaba (OPZ) tanpa menghasilkan dana maka tidak ada sumber dana yang dihasilkan. Sehingga apabila sumber daya sudah tidak ada maka

Berdasarkan Surat Penetapan Pemenang Nomor : 44.i /POKJA /ESDM-SRL/2012 tanggal 15 Agustus 2012, dengan ini kami Pokja Konstruksi pada Dinas ESDM Kabupaten

[r]

RKB Ponpes Salapul Muhajirin Desa Bukit Murau pada Dinas Pendidikan Kabupaten Sarolangun Tahun Anggaran 2012, dengan ini diumumkan bahwa :.. CALON

Bertitik tolak dari latar belakang pemikiran tersebut di atas, maka masalah yang sangat pundamental diteliti dan dibahas dalam rangkaian kegiatan penelitian ini

[r]