08832323.2013.778806

(1)

Full Terms & Conditions of access and use can be found at

http://www.tandfonline.com/action/journalInformation?journalCode=vjeb20

Download by: [Universitas Maritim Raja Ali Haji] Date: 11 January 2016, At: 20:36

Journal of Education for Business

ISSN: 0883-2323 (Print) 1940-3356 (Online) Journal homepage: http://www.tandfonline.com/loi/vjeb20

Assurance of Learning and the MFT: Closing the

Loops With an Online Review

Karen L. Middleton & Karen A. Loveland

To cite this article: Karen L. Middleton & Karen A. Loveland (2014) Assurance of Learning and the MFT: Closing the Loops With an Online Review, Journal of Education for Business, 89:3, 126-132, DOI: 10.1080/08832323.2013.778806

To link to this article: http://dx.doi.org/10.1080/08832323.2013.778806

Published online: 06 Mar 2014.

Submit your article to this journal

Article views: 56

View related articles

(2)

ISSN: 0883-2323 print / 1940-3356 online DOI: 10.1080/08832323.2013.778806

Assurance of Learning and the MFT: Closing the

Loops With an Online Review

Karen L. Middleton and Karen A. Loveland

Texas A&M University–Corpus Christi, Corpus Christi, Texas, USA

The authors describe the four stages of the closing the loop process undertaken by a college of business (COB) over a 6-year period. The COB developed and offered an online, noncredit review course to help students prepare for the Major Field Test in Business (MFT). Early results demonstrated the efficacy of the course as student scores rose from the 40th percentile to the 65th percentile. The authors analyze stage 4 data to determine the relationship between the online review and MFT performance. Results suggest the impact of the online review course diminishes as overall performance increases. Assessment outcomes may be influenced by the number of Hispanic and international students taking the MFT.

Keywords:assessment, closing the loop, Major Field Test, MFT, online review course, student learning

Many schools in the United States are under increasing pres-sure to show learning goal attainment through assessment testing (Smith, 2011). The No Child Left Behind Act of 2001 (2002) requires assessment testing for primary and secondary schools. Governing boards are holding the universities in their systems accountable for providing evidence of retention and graduation rates as well as licensing attainment following graduation (e.g., Cigarroa, 2011; Hammond, 2011; Ludwig, 2011). Accrediting agencies such as Association to Advance Collegiate Schools of Business International (AACSB) and the Southern Association of Colleges and Schools (SACS) are requiring professional colleges to provide assurance of learning (AOL) standards during their intensive accredita-tion processes (AACSB, 2004; SACS, 2009). For example, AACSB International accredited colleges of business (COBs) are required to develop AOL standards to measure students’ learning and retention of business skill sets, and diagnose students’ ability to apply the skill sets in a business context (AACSB, 2004).

AACSB standards require that a process be employed by COBs to collect assessment information that can then be used to continuously improve courses and programs, a process that was labeled closing the loop (Bisoux, 2008). The AACSB

Correspondence should be addressed to Karen A. Loveland, Texas A&M University-Corpus Christi, Department of Management and Mar-keting, 6300 Ocean Drive, Corpus Christi, TX 78312–5808, USA. E-mail: [email protected]

(2013) recommends a series of five steps in the process: (a) define student learning goals and objectives; (b) align cur-ricula with the adopted goals; (c) identify instruments and measures to assess learning; (d) collect, analyze, and dis-seminate assessment information; and (e) use the assessment information for continuous improvement.

One of the most popular assessment tools used to as-sess learning in a business degree program is the Educa-tional Testing Service’s (ETS) Major Field Test in Business (MFT) (Bagamery, Lasik, & Nixon, 2005; Bycio & Allen, 2007; Martell, 2007). The MFT for the bachelor’s degree in business measures the “Common Body of Knowledge” for undergraduate business courses. The test incorporates the following topic areas included in the business core for most institutions offering business degrees: accounting, eco-nomics, finance, business law, management, marketing, and quantitative analysis. It also covers social and international considerations as recommended by business accrediting agencies.

The MFT in business contains 120 multiple-choice ques-tions designed to measure students’ ability to retain and apply the critical facts, concepts, theories, and analytical meth-ods learned in their core business classes. Students typically take the test during their final year of coursework after they have completed all the required core courses in business. The ETS provides participating institutions with individual student scores (e.g., total score, scaled subscores) and ag-gregate institutional mean scores (e.g., total scores, scores

(3)

ASSURANCE OF LEARNING AND THE MFT 127

for each assessment indicator). The ETS provides additional percentile scores to allow institutions to compare the

perfor-mance of their students to all the 400+institutions

adminis-tering the MFT.

BACKGROUND

Following the five step process outlined previously, the sub-ject COB faculty developed a set of learning obsub-jectives for the undergraduate business curriculum and aligned the busi-ness curriculum with those objectives. They identified the MFT as one measure of overall learning and began admin-istering the test to students during the last semester of their senior year. The initial MFT goal targeted aggregate scores in the 50th percentile for each administration of the test. Stu-dents scored at the 40th percentile on the first administration, prompting the COB to begin looking for ways to close the loop and improve student performance.

The COB assessment committee (AC), working with the instructors for the capstone business course, sought ways to increase MFT scores. They offered students 50 points as a motivator for simply taking the exam. The motivator failed to motivate, and the mean scores did not improve the following term. The capstone instructors then took over the administra-tion of the MFT from the AC and sought to professionalize its administration. They redesigned the process to manage students’ preparation for the exam and strengthen students’ perceptions of its importance to their future. They also began collecting information from students about their experiences with the MFT. Student scores showed improvement over the next two terms, but aggregate scores still failed to achieve the 50th percentile objective.

Evidence collected from the student surveys suggested that students struggled with questions related to topics learned in courses taken two or more years before they sat for the MFT. Closer examination of the individual assessment in-dicators revealed that student performance in economics and accounting, courses typically completed during a student’s sophomore year, was especially low. The AC speculated that giving students the chance to review core class material be-fore sitting for the MFT might improve student performance.

CLOSING THE LOOP: DEVELOPMENT OF THE MFT REVIEW COURSE

The AC focus was now on providing students with the op-portunity to review terms, concepts, and analytical methods learned in the required core courses. The committee con-sidered several methods for presenting this opportunity to students. One option was to ask the capstone instructors to review key concepts from each functional area at the be-ginning of the term. The AC rejected this idea due to time constraints on the content coverage already required in the

course. The second option was to develop a series of short seminars taught by core course instructors. The AC rejected this idea when it became clear that most students did not have the time or the motivation to attend scheduled review sessions during their last term. The third and final option was to develop a review course that incorporated practice tests in each of the major core disciplines examined in the MFT. The AC chose to offer the review course as an online non-credit course for several reasons: (a) it allowed flexibility of access from anywhere at any time thereby maximizing the number of students who could benefit from the course; (b) it allowed the COB to track student interaction with the course and collect data for the assessment process without creating significant workload for the committee; and (c) it spread out the workload related to course development across a larger percentage of the COB faculty.

A faculty member familiar with the university’s online course management software developed the online review course template. Faculty from each of the seven content areas covered by the MFT developed a set of review slides and a test bank of multiple choice questions based on the content outline provided by the ETS. The course contained seven review modules. Each review module contained the review slides and a practice quiz that randomly selected 20 questions from the question database for that module. Students could take each review quiz as many times as they liked to review the material and prepare for the MFT.

Closing the Loop: Round 1

The AC made the MFT review course available to students for the first time in the spring of 2007. Students registered for the review course when they registered for the capstone course. Student had access to the course from the third week of the term until the day before MFT administration. The capstone instructors repeatedly encouraged students to re-view the slides and take the quizzes to assess their readiness for the MFT.

The first term, 50% of the student accessed the course materials and approximately 25% of the students completed at least one review quiz with a passing score of 70% or better. The students achieved the 50th percentile objective for the MFT that term despite the low participation rate in the review. Further analysis of the data revealed a positive correlation between participation in the review course and MFT performance.

Preliminary results encouraged the AC to seek additional methods for increasing student MFT performance. The cap-stone instructors added new points to their classes to motivate completion of the review quizzes and performance on the MFT. Students needed to score 70% or better on the review quiz in order to achieve a “pass” for each content module.

(4)

TABLE 1

Points Awarded for Successful MFT Review Modules Passed

Modules passed at 70% Credit Total points awarded

Seven modules 100% 50 points awarded

Six modules 85% 43 points awarded

Five modules 70% 35 points awarded

Less than five modules 0% 0 points awarded

Note: MFT=Major Field Test.

Capstone instructors awarded points based on the percentage of successful passes (see Table 1).

Students also received up to 50 points based on their MFT percentile score as shown in Table 2. Students could earn up to 100 points in total during the semester for their perfor-mance on the review and the MFT.

The new points spurred student efforts and the average scores increased to the 55–60th percentile range on the next two administrations of the test.

The AC then sought to encourage further practice and repeti-tion of the review test modules to raise the MFT scores above the 60th percentile. Accounting professors re-examined their review slides and question database for accounting topic cov-erage. They revised the accounting review quizzes to include question sets that ensured each attempt at the quiz contained a representative mix of accounting topics questions. Follow-ing positive student response and improved aggregate perfor-mance on the accounting assessment indicator the next term, professors in the other disciplines followed suit creating new databases with question sets covering required learning top-ics. Results from the summer of 2008 through the spring of 2009 further demonstrated the effectiveness of the MFT review course with student scores averaging at or above the 60th percentile.

A study of the results from the round 3 closing the loop actions showed overlearning (repeated attempts at a review quiz after achieving a passing score of 70%) had a posi-tive impact on transfer of learning to the MFT (Middleton & Loveland, 2009). Contrary to expectations, practice (mea-sured by the number of review quizzes attempted) and

mas-TABLE 2

Points Awarded for MFT Percentile Score

MFT percentile score Credit Total points awarded

75–100 Percentile 100% 50 points awarded

Less than 25 percentile 0% 0 points awarded

Note: MFT=Major Field Test.

tery of learning (measured by the number of review modules passed and the number of attempts required to pass) related negatively to transfer of learning. The researchers specu-lated that the measures of practice and transfer of learn-ing were actually measurlearn-ing prior learnlearn-ing. Students who mastered the material during the original core course ap-peared to require fewer attempts to achieve the passing stan-dard. Other students gamed the system by attempting each quiz a large number of times while they memorized cor-rect answers to achieve the required passing score on later attempts.

Another limitation of the third round of closing the loops analyses was that it did not include controls that have been found to be predictors of MFT outcomes. Previous studies found positive relationships between SAT verbal and math scores, grade point averages (GPA) in the major area of con-centration, overall university GPA and performance on the MFT (Bagemery et al., 2005; Bycio & Allen, 2007). Other research has found that ethnicity may act as a barrier to effec-tive learning (Wilson, 2001). This may be particularly true of online learning experiences for these ethnic groups (Muilen-burg & Berge, 2005). These findings led to the fourth stage of the closing the loop process.

The AC made several changes to the MFT administration and review course usage during the fourth stage. The committee relieved the Capstone professors of test administration duties, but asked them to retain the review and MFT points in their syllabi. The AC made the course available on the first day of classes and the passing standard for the MFT review modules increased to 80%. The points awarded remained the same as shown in Table 1. Students who scored in the top 25th percentile received a letter of congratulations from the dean for their portfolios.

The remainder of this paper focuses on the exploratory study of the outcomes of round 4 of the closing the loop process. We seek to answer four research questions.

RESEARCH QUESTIONS

Research Question 1

Studies suggest that when an individual recalls a previous learning experience, the retrieval of the experience results in an enhanced memory of the encoded contents (Benjamin & Tullis, 2010; Roher, Taylor, & Sholar, 2010). Successful retrieval from memory produces a stronger learning effect than does an additional presentation of the same information (Karpicke & Roediger, 2010). Retrieval practice necessary for successfully completing required testing becomes a pow-erful tool for increasing the retention of classroom learning. Other research suggests that learners who must make a con-centrated effort to retrieve and process the material are also

(5)

more likely to transfer the learned material to later test ques-tions that require them to make more complex inferences (Kang, McDermott, & Roediger, 2007; Karpicke & Blunt, 2011). The deeper the learner must go to retrieve the en-coded learning, the stronger the effect on permanent learning (Pyc & Rawson, 2009).

The opportunities for retrieval practice combined with the new higher standard for passing (mastering) the MFT review led to the first research question:

Research Question 1 (RQ1):Does passing the MFT review course have a positive impact on MFT performance?

Research Question 2

Studies have shown that repeated tests lead to greater learn-ing (Kornell, Bjork, & Garcia, 2011; Roediger & Karpicke, 2006). Repeated tests increase retrieval strength via addi-tional exposure to the encoded material (Karpicke & Roedi-ger, 2010) and enhance memory, which generally improves performance in standard recall (Hintzman, 2010). Recursive reminding operates to recover immediate recency (practice for the test) and embedded recency (learning that has pre-viously occurred in the classroom) (Hintzman, 2010). The benefits of overlearning persist when practice occurs across multiple sessions rather than cramming the same amount of learning into one practice session. This expanded retrieval practice enhances long-term retention of learning (Storm, Bjork, & Storm, 2010).

The third round study (Middleton & Loveland, 2009) found a positive correlation between repeated practice in the MFT review course and MFT performance, leading to the second research question:

RQ2:Does repeated practice in the MFT review course have

a positive impact on MFT performance?

Research Questions 3 and 4

To address the limitations of the third round study, we in-cluded the last two research questions:

RQ3:Do students of different demographic backgrounds

per-form differently on the MFT?

RQ4:Do students with different levels of academic potential

and achievement perform differently on the MFT?

METHODS

The sample consisted of 372 students who took the MFT during the 2010 and 2011 calendar years. We obtained MFT scores from the AC and extracted the total number of review modules passed (RQ1) and the total number of review quizzes each student completed after achieving a passing score (RQ2) from the MFT review course. We then obtained the follow-ing demographic data from university records: SAT scores

(verbal, math, and combined scores), students’ overall GPA at graduation, and students’ GPA in their major (RQ3); and gender, age, and ethnicity (RQ4). We also identified students that the university classified as international students, regard-less of ethnicity. We used ordinary least squares regression in SPSS (IBM SPSS Statistics version 20) to analyze the data. The study used an alpha of .10 (two-tailed test) as the standard for statistical significance as befits the exploratory nature of the research.

RESULTS AND DISCUSSION

The average MFT score in the study sample was 156.92 (60th percentile) with a low score of 125 and a high score of 190. The average age of the sample was 26.1 years old with a low of 21 years old and a high of 56 years old. Males comprised 55.1% of the sample (205 students). The ethnic composi-tion of the sample was the following: 50.3% White (non-Hispanic), 34.4% Hispanic, 4.4% Asian–Pacific Islander, and 2.2% Black with the remaining 8.6% classified as interna-tional students. The students in the sample had an average

overall GPA of 3.16 (range=1.80–4.0) and an average GPA

in major classes of 2.98 (range=2.14–4.0). SAT data were

available for 206 students (55.4% of the sample). The

aver-age SAT combined score was 998.8 (range=620–1420), the

average SAT verbal score was 487.9 (range=210–710), and

the average SAT math score was 511.0 (range=340–790).

The total attempts after passing ranged from zero to 74 attempts with a mean of 3.18 attempts per student. The total number of review modules passed ranged from zero to seven with 65.1% passing all seven modules, 7.5% passing six modules, 3.5% passing five modules, and 25% passing four or fewer modules.

Initial analysis involved bivariate regression using MFT score as the dependent variable. Selected results appear in Table 3.

The first research question asked if passing the MFT re-view course had an impact on MFT performance. The bivari-ate results for the total number of modules passed were not statistically significant indicating that mastering the material in the review modules did not lead to higher MFT scores. A possible explanation for the insignificant correlation be-tween modules passed and MFT score is that the learning encouraged and developed by mastering the MFT review course does not transfer to the MFT. The questions used in some or all of the review modules may not accurately reflect the content or structure of the questions on the MFT. Suc-cessful completion of the MFT review course may actually be having a detrimental impact on MFT performance. The negative beta, while not statistically significant, may indicate that successful completion of the review may generate irrele-vant or inaccurate knowledge. This may be especially true if the structure and content of the current MFT review course is not consistent with the MFT itself. It follows that inaccurate

(6)

TABLE 3

Bivariate Regression Results for Study Measures

Measure n Beta R2 _{Sig. (2-tailed)}

Total modules passed 372 −0.303 .002 .347

Note: Dependent variable was Major Field Test score. SAT=Scholastic Aptitude Test; GPA=grade point average; ESLI=English as a Second Language International.

student expectations following a successful review process could have a negative impact on MFT performance.

The second research question asked if repeated practice in the MFT review course had an impact on MFT performance. The bivariate results indicate that there was no significant relationship between repeated attempts after passing the re-view quizzes and MFT performance. This result is surprising because the earlier study (Middleton & Loveland, 2009) re-ported that students who continued to attempt quizzes after achieving a passing score increased their MFT score by an average of 1.3 points for each extra attempt they completed. In the present study, this beta was negative, suggesting that while the difference was small and not statistically signifi-cant, completing extra attempts after passing a review quiz might have decreased MFT performance.

The third research question asked if the demographic background of students had an effect on MFT performance.

The results for gender were statistically significant (p=.000)

and indicated that male students scored an average of 6.2 points higher on the MFT than female students. Gender ex-plained nearly 6% of the variation in MFT scores. This re-sult is consistent with prior research (e.g., Contreras, Badua, Chen, & Mitchell, 2011). The regression results for age were not statistically significant indicating that student age had no impact on MFT performance. This result was surprising be-cause previous research has identified a positive correlation between age and testing performance in general and MFT performance specifically (e.g., Contreas et al., 2011).

The most disturbing of the results was that the impact of Hispanic origin was statistically significant explaining 7.6% of the variation in MFT performance. Students of Hispanic origin in the present study scored nearly seven points lower on the MFT than non-Hispanic students. These results suggest the possibility of ethnic bias in the MFT that has a differential impact on the accreditation-related assessment of institutions that are designated Hispanic serving institutions (HSI) such as the university in this study.

The study sample also included 32 students classified as English as a Second Language International students. It was also troubling to note that the international students scored nearly 10 points lower on the MFT. This result suggests a bias similar to the previously noted Hispanic bias may also exist at universities that serve a disproportionate number of international students.

The final research question asked if students with differ-ent levels of academic potdiffer-ential and achievemdiffer-ent performed differently on the MFT. All three measures of academic po-tential at entry were statistically significant and consistent with prior research. Combined SAT score was the strongest measure in the study, explaining 41.2% of the variation in sample. It is interesting to note that SAT verbal scores ex-plained a larger percentage of the variation in MFT scores than SAT math scores in the study sample. Some other studies have suggested that the MFT favors quantitative over verbal skills. Curiosity about the inconsistency prompted us to con-duct a stepwise regression of the three SAT score variables against MFT score. The results included SAT score in the model and excluded both SAT verbal and math scores. As expected, the SAT verbal scores exhibited a moderate level of

multicollinearity (tolerance=.188, VIF=5.308). SAT math

scores also exhibited a moderate level of multicollinearity

(tolerance=.192, VIF =5.213). Examination of the

par-tial correlations indicated that SAT verbal and SAT math had identical levels of correlation with MFT score (.108), but that the correlation for SAT math was negative. The combined re-sults suggest that, for this study sample at least, verbal skills were a more important determinant of MFT score.

Both measures of academic achievement were also sta-tistically significant and consistent with prior research. Of the two measures, GPA in major was a stronger indicator of MFT performance explaining 12.3% of the variation in scores. This result is consistent with prior research and has strong face validity in that the MFT measures subject-specific transfer of learning in business courses.

The statistical significance of the bivariate results for eth-nicity combined with the conflict related to the lack of sig-nificance related to age, prompted us to dig a little deeper by doing stepwise multiple regression using all the variables in Table 3. The final model consisted of four variables that combined to explain nearly 50% of the variation in MFT scores: SAT score, overall major GPA, gender, and age. In

the multivariate model, age was statistically significant (p=

.029) with student MFT performance increasing by nearly .7 points for each year a student increased in age. More im-portantly, the multivariate model excluded the ethnicity and international variables, alleviating at least some of the initial concern of testing bias.

CONCLUSIONS AND IMPLICATIONS

This exploratory study makes three primary contributions to the existing literature. First, we describe four closing the

(7)

loop stages employed to support the AOL process. Then we outlined the activities used at each stage to develop a review and practice course to assist students with their preparation for the MFT. Second, we analyzed the most recent outcomes of the MFT review course and identified several important considerations for COBs who use the MFT and a related review course as part of their AOL program. Finally, we identified a potential source of differential performance on the MFT related to student demographics.

It may be tempting to conclude that the opportunities for review, practice, and repetition offered by the development of an MFT review course have no impact on student per-formance on the MFT. We would caution against such a conclusion for several reasons.

Several unpublished reports by the subject COB docu-mented the positive impact of the MFT review course used in this study. Before the creation of the online review course in 2007, COB students consistently scored well below the 50th percentile on the MFT and the trend line was close to flat, indicating no improvement over time in student perfor-mance on this assessment measure.

MFT scores increased to nearly the 60th percentile the first semester students used the course without grade-related rewards and the correlations between MFT score and review, mastery of learning, and repetition were statistically signif-icant. The trend line began to show steady improvement in MFT scores over the next two years. The AC attributed much of this improvement to the evolving MFT review course.

A prior study using Fall 2008 data revealed that the im-plementation of grade-related rewards for performance on the MFT review course and on the MFT itself had started to dilute the value of the MFT review course as an opportunity to improve student performance on the MFT. The first term included in this study, the Spring 2010 term, students’ scores increased to the 75th percentile. At this level of average per-formance, completing the MFT review course ceased to have a significant correlation with MFT performance as the data in this study revealed.

Note that the development and implementation of the MFT review course was not the only strategy the COB em-ployed over the past 6 years to improve MFT scores. For one, the COB added a required statement to the syllabi of all core business courses noting the importance of the MFT and the MFT review course and advising students to retain their textbooks and class notes to prepare for the MFT during their senior year. Additional efforts to improve the content and structure of the review course involved a large percent-age of the COB faculty. This involvement led to a heightened awareness of the MFT as an assessment tool and likely con-tributed to enhanced the coverage of MFT-related concepts and theories in all the core courses. An increasing propor-tion of students taking the MFT derived less benefit from the opportunity to review material they committed to memory earlier in their academic careers as a result. Future research

might attempt to control for the impact of heightened aware-ness and attention to MFT preparation by tracking results at an institution that has just started using the MFT as an assessment variable.

It is also possible that the content of the review course used in this study does not accurately represent the breadth or depth of coverage of the MFT as suggested previously. The subject COB should investigate the possibility of con-tent and/or implementation errors as part of its continuous improvement process. The broader implication of the appar-ent decline in the value of the MFT review course is that other institutions using, or considering the implementation of, a review course as part of their own assessment process should pay careful attention to both the content and the de-sign of the course. Overspecializing the course (e.g., mapping review questions to specific MFT questions), providing too much feedback, or failing to control for a variety of ways students may game a review course that offers grade-related rewards could all cause an institution to abandon a continu-ous improvement opportunity that is actually contributing to success of some students.

The findings in this study related to the impact of ethnic origin should raise concerns for COBs that serve a signif-icant number of Hispanic or international students. COBs that use the MFT for assessment need to be aware of the potential for a systematic disadvantage in comparative data if they serve a larger percentage of Hispanic or international students than peer institutions. Further researchers should in-vestigate whether an ethnic or cultural bias exists in the MFT itself.

The most significant limitation of this study was our in-ability to distinguish among many potential sources of vari-ation in MFT performance. The four significant variables noted previously (SAT score, GPA in major, gender, and age) explained less than 50% of the variation in MFT scores. None of those variables lend themselves to direct interven-tion as part of a continuous improvement plan related to the use of MFT scores as an assessment measure in accreditation processes. The most important conclusion regarding the var-ied results of this study is that further research is needed to identify activities COBs can use to close the loop in assess-ment reports related to MFT performance. What can we do to satisfy the requirements of accrediting bodies and legisla-tors using MFT scores to assess quality of learning if giving students the opportunity to review and practice what they learned in their core courses does not improve MFT perfor-mance? How should we structure opportunities to maximize the benefits for the greatest number of students if practice does improve performance?

The answers to these questions will become even more important in 2015 when the 15 new standards proposed by the AACSB (2013) go into effect. The standards, comprising 12 modified standards and three new standards, will put COBs under increasing pressure to promote and uphold innovation, impact, and engagement. The need to tie the desired learning

(8)

objectives with the measured outcomes will require critical problem solving in the closing the loops process.

REFERENCES

Association to Advance Collegiate Schools of Business (AACSB). (2004). Eligibility procedures and accreditation standards for business accredi-tation. St. Louis, MO: Author.

Association to Advance Collegiate Schools of Business (AACSB). (2013). AACSB assurance of learning standards: An interpretation (AACSB White Paper No. 3 issued by AACSB International Accreditation Co-ordinating Committee and AACSB International Accreditation Quality Committee). St. Louis, MO: Author.

Bagamery, B. D., Lasik, J. J., & Nixon, D. R. (2005). Determinants of success on the ETS business major field exam for students in an undergraduate multisite regional university business program.Journal of Education for Business,81, 55–63.

Benjamin, A. S., & Tullis, J. (2010). What makes distributed practice effec-tive?Cognitive Psychology,61, 228–247.

Bisoux, T. (2008, March/April). Measuring success.BizEd, 22–30. Bycio, P., & Allen, J. S. (2007). Factors associated with performance on

the Educational Testing Service (ETS) Major Field Achievement Test in Business (MFAT-B).Journal of Education for Business,82, 196–201. Cigarroa, F. G. (2011, May 6). UT system a leader in

accountabil-ity.Houston Chronicle. Retrieved from http://www.chron.com/default/ article/University-of-Texas-System-a-leader-in-1605491.php

Contreras, S., Badua, F., Chen, J. S., & Mitchell, A. (2011). Documenting and Explaining Major Field Test Results Among Undergraduate Students. Journal of Education for Business,86, 64–70.

Hammond, B. (2011, July 28). Focus on outcomes of higher educa-tion. Corpus Christi Caller Times. Retrieved from http://www.caller. com/news/2011/jul/28/focus-on-outcomes-of-higher-education/ Hintzman, D. L. (2010). How does repetition affect memory? Evidence from

judgments of recency.Memory & Cognition,38, 102–115.

Kang, S. H. K., McDermott, K. B., & Roediger, H. L. III. (2007). Test format and corrective feedback modify the effect of testing on long-term retention.European Journal of Cognitive Psychology,19, 528–558. Karpicke, J. D., & Blunt, J. R. (2011). Retrieval practice produces more

learning than elaborative studying with concept mapping. Science, 331(6018), 772–775.

Karpicke, J. D., & Roediger, H. L. III. (2010). Is expanding retrieval a superior method for learning text materials?Memory & Cognition,38, 116–124.

Kornell, N., Bjork, R. A., & Garcia, M. A. (2011). Why tests appear to prevent forgetting: A distribution-based bifurcation model.Journal of Memory and Language,65, 85–97.

Ludwig, M. (2011, June 30). No crisis in higher ed, says depart-ing A&M chancellor. San Antonio Express News. Retrieved from http://www.caller.com/news/2011/jul/28/focus-on-outcomes-of-higher-education/

Martell, K. (2007). Assessing student learning: Are business schools making the grade?Journal of Education for Business,82, 189–195.

Middleton, K. L., & Loveland, K. A. (2009, April).The relationship be-tween online review practice and performance outcomes in the MFT administration process. Paper presented at the Society for Advancement of Management Annual Conference, Las Vegas, NV.

Muilenburg, L. Y., & Berge, Z. L. 2005. Student barriers to online learning: A factor analytic study.Distance Education,26, 29–48.

No Child Left Behind Act of 2001, Pub. L. No. 107-110,§115, Stat. 1425 (2002).

Pyc, M. A., & Rawson, K. A. (2009). Testing the retrieval effort hy-pothesis: Does greater difficulty correctly recalling information lead to higher levels of memory?Journal of Memory and Language,60, 437– 447.

Roediger, H. L. III, & Karpicke, J. D. (2006). The power of testing memory: Basic research and implications for educational practice.Perspectives on Psychological Science,1, 181–210.

Roher, D., Taylor, K., & Sholar, B. (2010). Test enhance the transfer of learning.Journal of Experimental Psychology: Learning, Memory, and Cognition,36, 233–269.

Smith, M. (2011, July 24). Change in rating formula creates anxiety in schools.The New York Times. Retrieved from http://www.nytimes. com/2011/07/24/us/24ttaccountable.html? r=0

Southern Association of Colleges and Schools (SAC). (2009).The principles of accreditation: Foundations for quality enhancement (4thed.). Decatur, GA: Author.

Storm, B. C., Bjork, R. A., & Storm, J. C. (2010). Optimizing retrieval as a learning event: when and why expanding retrieval practice enhances long-term retention.Memory & Cognition,38, 244–253.

Wilson, M. S. (2001). Cultural considerations for online instruction and learning.Distance Education,22, 52–64.