Full Terms & Conditions of access and use can be found at
http://www.tandfonline.com/action/journalInformation?journalCode=vjeb20
Download by: [Universitas Maritim Raja Ali Haji] Date: 11 January 2016, At: 22:04
Journal of Education for Business
ISSN: 0883-2323 (Print) 1940-3356 (Online) Journal homepage: http://www.tandfonline.com/loi/vjeb20
The GMAT as a Predictor of MBA Performance:
Less Success Than Meets the Eye
Darrin Kass , Christian Grandzol & William Bommer
To cite this article: Darrin Kass , Christian Grandzol & William Bommer (2012) The GMAT as a Predictor of MBA Performance: Less Success Than Meets the Eye, Journal of Education for Business, 87:5, 290-295, DOI: 10.1080/08832323.2011.623196
To link to this article: http://dx.doi.org/10.1080/08832323.2011.623196
Published online: 05 Jun 2012.
Submit your article to this journal
Article views: 248
View related articles
ISSN: 0883-2323 print / 1940-3356 online DOI: 10.1080/08832323.2011.623196
The GMAT as a Predictor of MBA Performance:
Less Success Than Meets the Eye
Darrin Kass and Christian Grandzol
Bloomsburg University of Pennsylvania, Bloomsburg, Pennsylvania, USA
William Bommer
Fresno State University, Fresno, California, USA
Consistent with previous research, the authors found that the combined use of undergradu-ate grade point average and the Graduundergradu-ate Management Admission Test (GMAT) verbal and quantitative sections successfully predicted performance in a master of business administra-tion (MBA) program. However, these measures did not successfully predict the competencies that underlie managerial effectiveness, including communication, teamwork, decision making, leadership initiative, and planning and organizing. These competencies were better predicted, albeit imperfectly, by the analytical writing section of the GMAT. These findings are discussed with regard to the curriculum of MBA programs.
Keywords: GMAT, MBA, managerial competencies, validity
The Graduate Management Admission Test (GMAT) is a standardized test used internationally by over 4,500 gradu-ate management programs at 1,900 schools to make admis-sion deciadmis-sions (Graduate Management Admisadmis-sions Council, 2010). GMAT admission criteria are utilized by national me-dia, such as the Wall Street Journal, US News and World Report, andBusiness Week, as measures of school selectiv-ity. These measures play a significant role in the reputation of colleges and universities (Wright & Bachrach, 2003).
There has been considerable support for the validity of the GMAT to predict the academic performance of students in master of business administration (MBA) programs (e.g., Kuncel, Crede, & Thomas, 2007; Oh, Schmidt, Shaffer, & Le, 2008). A recent meta-analysis, using a corrected mean correlation, found that the validity of GMAT to predict grad-uate academic performance was .50 for first year gradgrad-uate grade point average (GPA) and overall graduate GPA (Oh et al.). Studies have shown that the test was equally valid for non-U.S. students (Koys, 2005), executive MBA stu-dents (Siegert, 2008), and doctoral business stustu-dents (Siegert, 2007). Additionally, there appeared to be no predictive dif-ferences based on students’ gender or race and ethnicity
Correspondence should be addressed to Darrin Kass, Bloomsburg Uni-versity of Pennsylvania, Department of Management, 400 E. Second Street, Bloomsburg, PA 17815, USA. E-mail: dkass@bloomu.edu
(Graduate Management Admissions Council, 2010; Sireci & Talento-Miller, 2006).
The GMAT consists of three sections: quantitative (GMAT-Q), verbal (GMAT-V), and analytical writing (AWA); however, the majority of studies examined only the verbal and quantitative sections (e.g., Kuncel et al., 2007; Oh et al., 2008). The combined use of verbal and quantitative scores had greater validity than undergraduate GPA (UGPA) alone, and the best predictive validity was found when ver-bal, quantitative, and UGPA were combined in the admission decision (Talento-Miller & Rudner, 2005). Studies that have included the analytical writing sections demonstrated that their predictive utility is marginal, accounting for a non-significant 1% of the variation in graduate GPA above and beyond UGPA and the GMAT quantitative and verbal sec-tions (Sireci & Talento-Miller). Researchers have reported that the writing sections have slightly better validity for non-U.S. test takers (Talento-Miller, 2005; Talento-Miller & Rudner).
The GMAT’s high level of validity led some authors to conclude that the GMAT is a good indicator of future job performance (e.g., Kuncel et al., 2007; Oh et al., 2008) be-cause research has shown a modest relationship between aca-demic success and job performance (Roth, BeVier, Switzer, & Schippmann, 1996). Kuncel et al. offered that the abili-ties required for success in school are very similar to those required for success at work, and that the academic work of
GMAT AND MBA PERFORMANCE 291
MBA programs and the work of managers are very similar in content (Kuncel et al.). Given this overlap, Kuncel et al. argued, “If the GMAT predicts student success it should also have a relationship with subsequent job success” (p. 55).
What is lacking is an examination of the relationship be-tween GMAT scores and the skills that underlie managerial effectiveness. Hoover, Giambatista, Sorenson, and Bommer (2010) noted, “The possession of a set of cognitive schemas is not the same thing as the possession of a set of behavioral repertoires” (p. 193). The Association to Advance Collegiate Schools of Business (AACSB; 2010) concurred, emphasizing that knowledge should not be the only goal of an MBA pro-gram. Instead, students must develop the capacities to lead, solve problems, and innovate. The GMAT was designed to predict graduate academic success, and it does so with strong accuracy, but it is questionable that it predicts the competen-cies that are necessary in a managerial position. GPA suffers the same doubt. Ultimately, the primary goal of most MBA programs is to develop the competencies necessary for suc-cess as a future manager (for a complete review, see Rubin & Dierdorff, 2011), and graduate school success should be evaluated in terms of the acquisition of these competencies. For a variety of reasons, GPA is not a perfect measure of graduate school success (Sireci & Talento-Miller, 2006) and it, too, may not reveal whether students have acquired these managerial competencies.
Our purpose was to determine if the GMAT sections and MBA GPA are valid predictors of future managerial perfor-mance. We used a managerial exercise known as Iliad to mea-sure various competencies such as leadership and decision-making. These are critical skills that MBA programs should be interested in developing and documenting. Do the ways re-searchers predict and measure graduate school performance (GMAT and MBA GPA) provide an accurate sense of a stu-dent’s managerial abilities?
METHOD
Participants
Data were collected from 72 students enrolled in an MBA program at one public university in Pennsylvania over the course of three academic years, from 2007 to 2010. The par-ticipants included 24 women and 48 men, whose ages ranged from 22 to 54 years, with an average age of 29.35 years . Ten of the 72 participants were non-U.S. students. The GMAT is required for admission of applicants and is combined with UGPA for admissions decisions. The managerial exercise is part of the MBAs assurance of learning program.
Measures
Managerial performance. The Iliad Assessment Cen-ter (Bommer & Bartels, 1996) was used to identify skill levels in five dimensions: active communication, teamwork,
decision making, leadership initiative, and planning and or-ganizing. These five skill dimensions map onto several of the six behavioral competencies that underlie any managerial oc-cupation, specifically managing human capital and managing decision-making processes (Rubin & Dierdorff, 2009). The Iliad Assessment Center has been validated and employed in other published studies and Rode et al. (2005) discussed it in greater detail.
The assessment center simulates a day in the life of a man-ager and lasts 145 min. Scores were derived from a series of behavioral objectives students completed during the assess-ment, including a managerial in-basket, two team meetings, and an individual speech. The team meetings and speech were videotaped, and all materials were evaluated by raters from the Iliad Assessment Center.
The raters were blind to this study and to the identity of the students (they were in different states and had no per-sonal connections). Raters had an average of 1 year of rating experience and were either present or former students in a master’s in industrial psychology program. Raters received frame of reference training and rated in pairs. Conflicts be-tween raters were settled through reviewing the recordings and reaching consensus on the presence, absence, and mag-nitude or effectiveness of the specific behaviors.
GMAT. The GMAT, developed and administered by the Educational Testing Service, measures basic verbal, math-ematical, and analytical writing skills. The GMAT is a computer-adaptive test with four timed sections. Quantita-tive consists of data sufficiency and problem solving. Verbal contains reading comprehension, sentence correction, and critical reasoning. Analytical writing consists of an analysis of an issue and an analysis of an argument.
Data Analysis
We ran several analyses to investigate the question under study. First, we ran a sequential regression to examine the validity of using UGPA and the three GMAT scores to predict GPA in our MBA program. Second, we ran correlations to examine the strength and direction of relationships between MBA GPA, GMAT scores, and the managerial competencies measured by Iliad. Finally, we executed stepwise regressions to determine which of these variables make meaningful con-tributions to the overall prediction of the management com-petencies.
RESULTS
We assessed the underlying assumptions of multiple regres-sion analysis using standard data screening protocols. No multivariate outliers were identified. We excluded three par-ticipants due to extreme values on the verbal or quantitative GMAT sections. Of the three outliers that were deleted, two were extremely high on the GMAT-Q and one was very low
TABLE 1
Means and Standard Deviations for Predictor and Dependent Variables
Leadership initiative 46.71 22.11
Decision making 143.65 23.45
Organizing 134.41 17.52
Communication 204.15 34.72
Teamwork 56.93 18.58
Note.UGPA=undergraduate grade point average; MBA=master of business administration; GPA=grade point average; GMAT=Graduate Management Admission Test; Q=quantitative; V=verbal; AWA= ana-lytical writing.
on the GMAT-V. This reduced the sample size to 69. Note that we excluded these data points because of the sensitivity of regressions to extreme values, but there were no meaningful differences in interpretation even when they were included. Though smaller than desired, the sample size met the mini-mum 15 per predictor for a reliable equation (Stevens, 2009). No marked violations of normality, homoscedasticity, or lin-earity were observed.
The mean scores and standard deviations for the vari-ables under investigation are reported in Table 1. The Iliad competencies are reported in raw score. The ranges of these scales vary, but in all cases, higher scores signify stronger performance.
We first examined which variables contribute to the pre-diction of MBA GPA using a sequential multiple regression. In this technique, various combinations of predictors are in-tentionally entered into the equation in blocks and the change inR2was evaluated to measure the value added of additional variables. We used this analysis to compare our sample to findings from a study involving several thousand students by Sireci and Talento-Miller (2006).
As shown in Table 2, Model 2, which included UGPA, GMAT-Q, and GMAT-V accounted for roughly 30% of the overall variance in MBA GPA. GMAT-Q and GMAT-V sig-nificantly increased the predication above and beyond UGPA (Model 1). In fact, adding those predictors nearly tripled the amount accounted for by UGPA alone; adding AWA into the equation (Model 3) did not significantly increase the predic-tion. These findings are nearly identical to those of Sireci and Talento-Miller’s (2006) large sample study in terms of predictive ability and value added of the various models. We believe this enhanced the confidence we can ascribe to our sample.
We next executed a correlation analysis to provide the direction and strength of the relationships between the indi-vidual variables of interest. As seen in Table 3, GMAT-V and
TABLE 2
Summary of Sequential Regression Analyses for MBA GPA
Model Predictor(s) R R2 Value added
1 UGPA .338 .114 .114∗
2 UGPA, GMAT-Q, GMAT-V .552 .305 .191∗
3 UGPA, GMAT-Q, GMAT-V, AWA
.569 .324 .019
Note.UGPA=undergraduate grade point average; MBA=master of business administration; GPA=grade point average; GMAT=Graduate Management Admission Test; Q=quantitative; V=verbal; AWA= ana-lytical writing.
∗p<.05.
AWA had significant and positive correlations with four of the five managerial competencies. MBA GPA had significant and positive correlations with two competencies. GMAT-Q was not significantly correlated with any of the competencies. Although correlations between variables are interesting, they do not provide an answer as to how well combina-tions of variables contribute to a prediction. For this, we executed stepwise regressions to see if MBA GPA, GMAT-Q, GMAT-V, AWA, or some combination of these reliably predicted the management competencies measured by Iliad. We chose to include all four potential predictors in Table 4 even if the predictor did not enter the equation; only the predictor that significantly entered into the model has a co-efficient displayed. With the exception of organizing, AWA was the only significant predictor of the various managerial competencies. Adding any other variable to the model after AWA was entered did not significantly add to the prediction of the competencies. GMAT-V was the only significant pre-dictor of organizing. Note that in all cases, the proportion of the competencies accounted for by the significant predictor was small and ranged between 7.1% and 13.9%.
DISCUSSION
Developing effective managers is a critical component of business education (e.g., Datar, Garvin, & Cullen, 2010;
TABLE 3
Correlation Matrix for Individual Variables
Variable
Note.MBA=master of business administration; GPA=grade point average; GMAT=Graduate Management Admission Test; Q=quantitative; V=verbal; AWA=analytical writing.
∗p<.05.
GMAT AND MBA PERFORMANCE 293
TABLE 4
Summary of Stepwise Regression Analyses
Standardized coefficients of predictors
Competency R R2 Intercept MBA GPA GMAT-Q GMAT-V AWA
Leadership initiative .319 .102 5.925 .319∗
Decision making .270 .073 107.050 .270∗
Organizing .266 .071 110.816 .266∗
Communication .373 .139 129.346 .373∗
Teamwork .317 .100 22.913 .317∗
Note.MBA=master of business administration; GPA=grade point average; GMAT=Graduate Management Admission Test; Q=quantitative; V= verbal; AWA=analytical writing.
∗p<.05.
Mintzberg, 2004; Rubin & Dierdorff, 2009, 2011). The 2002 report of the Management Education Task Force noted that, “collected evidence from business school alumni suggests that the most important predictor of business success is man-agement effectiveness” (p. 19). However, our results demon-strate that the metrics typically used to select and matriculate graduates (GMAT-Q+GMAT-V, MBA GPA) have little re-lationship with the competencies that underlie managerial effectiveness. In fact, GMAT-V predicted only one compe-tency. AWA predicted the other four. Ironically, the majority of validity studies have shown that the predictive utility of AWA for MBA GPA is low and nonsignificant, accounting for only 1–2% of variation after accounting for GMAT-V and GMAT-Q (e.g., Talento-Miller & Rudner, 2005).
An alarming finding of our study was that MBA GPA did not predict any of the managerial competencies on the IL-IAD assessment. We hypothesized that success in our MBA program would lead to the development of effective man-agement skills; our results indicate otherwise. This finding supports the criticisms regarding the relevance of MBA cur-riculum to the actual practice of management (e.g., Navarro, 2008; Pfeffer & Fong, 2002, 2003; Rubin & Dierdorff, 2009, 2011). Our results lend quantitative support to Mintzberg’s (2004) critique that MBA students “become knowledgeable about business, but remain untutored in the art and craft of management” (p. 79).
If grades earned in MBA courses assess achievement of isolated knowledge, where do students develop the necessary managerial competencies? While there is evidence that these skills can be learned through trial and error on the job (e.g., Tannenbaum, Beard, McNail, & Salas, 2009), MBA pro-grams, at a minimum, should enhance and expedite the devel-opment of these critical competencies. How do researchers know if the program improves the students’ performance of these competencies?
Our findings indicate that the GMAT (V and Q) and MBA GPA should not be used as indicators of future successful job performance. This is not entirely surprising; previous re-searchers have found that the relationship between grades and job performance was modest, .16, and was even lower
in business settings, .14 (Roth et al., 1996). As Mintzberg (2004) noted, “The GMAT constitutes a useful, but insuf-ficient screening device, more useful, in fact, to identify successful students than successful managers” (p. 15). A similar argument can be made for grades earned. Our re-sults demonstrated that academic success did not translate to higher performance on important managerial competencies. Our results are consistent with a growing body of literature that questions the ability of MBA programs to adequately pre-pare their students with the skills needed to succeed as man-agers in modern organizations (e.g., Mintzberg, 2004; Pfeffer & Fong, 2002; Rubin & Dierdorff, 2011). Specifically, the competencies reported as most critical for leadership effec-tiveness received the least amount of required coverage in MBA programs (Rubin & Dierdorff, 2009) and the cover-age of interpersonal, leadership, and communication skills were the “least effective components of business curricula” (Management Education Task Force, 2002, p. 19). Rynes, Trank, Lawson, and Iles (2003) referred to this problem as management education’s “legitimacy crisis” (p. 1).
To address this situation, some business educators have begun to emphasize the development of personal, interper-sonal, and group skills that underlie effective leadership (Whetten & Cameron, 2007). This requires a pedagogi-cal shift, from a focus on traditional cognitive-based learn-ing to skill-based learnlearn-ing, (Mintzberg, 1975; Whetton & Cameron), and from knowing to doing (Datar, Garvin, & Cullen, 2010). There has been empirical support for the ef-fectiveness of these types of curricular changes. Hoover et al. (2010) developed a pedagogical approach focused on devel-oping executive skills through cognitive, behavioral, and af-fective aspects of learning. This experiential learning model based on the concept of whole person learning was effective for improving managerial skills over the course of 5 years.
Our results show a potentially important direction for MBA curriculum, as the AWA was a predictor of 4 of the 5 managerial competencies assessed by the ILIAD: leadership initiative, decision making, teamwork, and communication. The AWA measures critical thinking and analysis, and the ability to coherently express ideas. Perhaps the skills that
underlie critical thinking and clear expression are similar to those required for effective management. More research is necessary before reaching a definitive conclusion, but the findings provide an avenue for improving MBA curriculum. In fact, Datar et al. (2011) identified critical thinking as one of the unmet curricular needs of MBA programs, and point to the Critical Analytical Thinking course at the Stanford Grad-uate School of Business as an example of how a curricular innovation can address this need.
Limitations
The study involved data from one school’s MBA program, with students that had a GPA and could take the Iliad as-sessment. Having data from only from one public university limits the generalizability of the results of our study. Addi-tionally, we did not assess all possible variables that may play a role in managerial competencies (i.e., experience).
The predictive validities of the GMAT for MBA GPA have exhibited wide ranges, from a low of –.45 to a high of .76 (Kuncel et al., 2007). This is likely due to various errors (sampling, etc.), which may also have affected our findings on managerial competencies. We also used the GMAT beyond its proposed use. The test is intended to predict students’ performance in graduate school, not necessarily managerial competencies. Finally, the study was subject to the universal problem of range restriction in educational research (Hunter, Schmidt, & Le, 2006). In our sample, there was a high mean for MBA GPA with a small standard deviation. There may not have been enough variability in MBA GPA to enable meaningful analysis on this measure.
Future Research
The study’s findings suggest potential research streams. Re-searchers with access to larger data pools can study the issue to address the generalizability of the results. Other com-monly used assessments of MBA school success such as the Educational Testing Service’s field test should be examined for relationships with managerial effectiveness. Ultimately, it would be useful for researchers to explore the metrics utilized to determine successful performance in an MBA program, as well as those used to select applicants into programs. As Ru-bin and Dierdorff (2009) noted, “If the primary intent of an MBA program is to develop future managers (or strengthen present managers’ capabilities), then the required curricula of such programs should represent an institution’s best attempt to capture the most essential content relevant to managerial work” (p. 211).
Conclusion
As previously reported in the literature and reinforced in this study, undergraduate GPA and the GMAT-V and GMAT-Q sections seem to differentiate MBA students’ performance when measuring the performance by the GPA students earn
in graduate school. These measures appear to be inadequate when measuring performance as a set of competencies re-quired to be a successful manager. The AWA section was a better, albeit far from complete, predictor of these compe-tencies. These findings signal the need for MBA programs to evaluate whether there is a disconnect between how they measure performance and their expectation to produce more effective managers.
REFERENCES
Association to Advance Collegiate Schools of Business. (2010).Eligibility procedures and accreditation standards for business education. Retrieved from http://www.aacsb.edu/accreditation/business/standards/
Bommer, W. H., & Bartels, L. K. (1996).The Iliad assessment center. Bloomington, IN: Tichenor.
Datar, S. M., Garvin, D. A., & Cullen, P. G. (2010).Rethinking the MBA: Business education at a crossroads. Boston, MA: Harvard Business Press. Datar, S. M., Garvin, D. A., & Cullen, P. G. (2011). Rethinking the MBA: Business education at a crossroads.Journal of Management Development, 30, 451–462.
Graduate Management Admissions Council. (2010).The GMAT exam. Re-trieved from http://www.gmac.com/gmac/about-the-gmat-exam.aspx Hoover, J. D., Giambatista, R. C., Sorenson, R. L., & Bommer, W. (2010).
Assessing the effectiveness of whole person learning pedagogy in skill acquisition.Academy of Management Learning and Education,9, 192– 203.
Hunter, J. E., Schmidt, F. L., & Le, H. (2006). Implications of direct and indirect range restriction for meta-analysis methods and findings.Journal of Applied Psychology,91, 594–612.
Koys, D. (2005). The validity of the Graduate Management Admissions Test for non-US students.Journal of Education for Business,80, 236– 239.
Kuncel, N. R., Crede, M., & Thomas, L. L. (2007). A meta-analysis of the predictive validity of the Graduate Management Admissions Test (GMAT) and Undergraduate Grade Point Average (UGPA) for graduate student academic performance.Academy of Management Learning and Education,6, 51–68.
Management Education Task Force. (2002).Management education at risk. Retrieved from http://www.aacsb.edu/publications/metf/metfreportfinal-august02.pdf
Mintzberg, H. (1975). The manager’s job: Folklore and fact.Harvard Busi-ness Review,53, 100–110.
Mintzberg, H. (2004).Managers not MBAS: A hard look at the soft prac-tice of management and management development. San Francisco, CA: Barrett-Koehler.
Navarro, P. (2008). The MBA core curricula of top-ranked U. S. business schools. A study in failure?Academy of Management Learning and Ed-ucation,7, 108–123.
Oh, I., Schmidt, F. L., Shaffer, J. A., & Le, H. (2008). The Graduate Man-agement Admission Test (GMAT) is more valid than we thought: A new development in meta-analysis and its implications for the validity of the GMAT.Academy of Management Learning and Education,7, 563– 570.
Pfeffer, J., & Fong, C. T. (2002). The end of business schools? Less success than meets the eye.Academy of Management Learning and Education,1, 79–95.
Pfeffer, J., & Fong, C. T. (2003). Assessing business schools: Reply to Connolly.Academy of Management Learning and Education,2, 368– 370.
Rode, J. C., Arthaud-Day, M. L., Mooney, C. H., Near, J. P., Baldwin, T. T., Bommer, W. H., & Rubin, R. (2005). Life satisfaction and student
GMAT AND MBA PERFORMANCE 295
performance.Academy of Management Learning and Education,4, 421– 433.
Roth, P. L., BeVier, C. A., Switzer, F. S., & Schippmann, J. S. (1996). Meta-analyzing the relationship between grades and job performance.Journal of Applied Psychology,81, 548–556.
Rubin, R. S., & Dierdorff, E. C. (2009). How relevant is the MBA? Assessing the alignment of required MBA curricula and required managerial com-petencies. Academy of Management Learning and Education,8, 208– 224.
Rubin, R. S., & Dierdorff, E. C. (2011). On the road to Abilene: Time to manage agreement about MBA curricular relevance.Academy of Man-agement Learning and Education,10, 208–224.
Rynes, S. L., Trank, C. Q., Lawson, A. M., & Iles, R. (2003).Behavioral coursework in business education: Growing evidence of a legitimacy crisis.Academy of Management Learning and Education,2, 269–283. Siegert, K. (2007).Predicting success in graduate management doctoral
programs. GMAC research reports. McLean, VA: Graduate Management Admission Council.
Siegert, K. (2008). Executive education: Predicting student success in Executive MBA programs. Journal of Education for Business, 83, 221–226.
Sireci, S. G., & Talento-Miller, E. (2006). Evaluating the predictive valid-ity of Graduate Management Admission Test scores.Educational and Psychological Measurement,66, 305–317.
Stevens, J. P. (2009).Applied multivariate statistics for the social sciences (5thed.). New York, NY: Routledge Academic.
Talento-Miller, E. (2005).GMAT validity for six European MBA programs. GMAC research reports. McLean, VA: Graduate Management Admission Council.
Talento-Miller, E., & Rudner, L. M. (2005).GMAT validity study summary report for 1997 to 2004. GMAC research reports. McLean, VA: Graduate Management Admission Council.
Tannenbaum, S. I., Beard, R. L., McNall, L. A., & Salas, E. (2009). Informal learning and development in organizations. In S. W. J. Kozlowski and E. Salas (Eds.),Learning, training, and development in organizations(pp. 303–332). New York, NY: Routledge.
Whetten, D. A., & Cameron, K. S. (2007).Developing management skills (7th ed.). Upper Saddle River, NJ: Prentice Hall.
Wright, R. E., & Bachrach, D. G. (2003). Testing for bias against females test takers of the Graduate Admissions Test and potential impact of admissions to graduate programs in Business.Journal of Education for Business,78, 324–328.
