Full Terms & Conditions of access and use can be found at
http://www.tandfonline.com/action/journalInformation?journalCode=vjeb20
Download by: [Universitas Maritim Raja Ali Haji] Date: 11 January 2016, At: 19:04
Journal of Education for Business
ISSN: 0883-2323 (Print) 1940-3356 (Online) Journal homepage: http://www.tandfonline.com/loi/vjeb20
Assessment at AACSB Schools: A Survey of Deans
Barbara M. Wheeling, Donald S. Miller & Thomas E. Slocombe
To cite this article: Barbara M. Wheeling, Donald S. Miller & Thomas E. Slocombe (2015) Assessment at AACSB Schools: A Survey of Deans, Journal of Education for Business, 90:1, 44-49, DOI: 10.1080/08832323.2014.973824
To link to this article: http://dx.doi.org/10.1080/08832323.2014.973824
Published online: 21 Nov 2014.
Submit your article to this journal
Article views: 84
View related articles
View Crossmark data
Assessment at AACSB Schools: A Survey of Deans
Barbara M. Wheeling
Montana State University Billings, Billings, Montana, USA
Donald S. Miller and Thomas E. Slocombe
Emporia State University, Emporia, Kansas, USAThe purpose of this research was to document the extent to which Association to Advance Collegiate Schools of Business (AACSB)–accredited business schools have implemented strategies to improve students’ ability to achieve program learning objectives. Assessment of academic programs is increasingly important at AACSB schools. Compliance with changing accreditation requirements and continuous improvement necessitated revision of assessment methods and modification of programs. Deans at 91 AACSB schools participated in this research. Communication skills were the most-assessed competency. Mandating increased coverage of specific skills or topics across the curriculum was the most frequent change made as a result of assessment. Further studies will be needed to track this dynamic activity.
Keywords: AACSB, accreditation, assessment
Optimizing students’ learning and performance has become increasingly significant in Association to Advance Colle-giate Schools of Business (AACSB)–accredited business schools, which are required to demonstrate commitment to continued assessment and improvement linked to that assessment (Pringle & Michel, 2007).
As accreditation policies evolved, accredited schools pursued a moving target, attempting to find more effective methods to improve performance and the best way to docu-ment these efforts (Kelley, Tong, & Choi, 2010). Although the 2003 standards of the AACSB emphasized the impor-tance of documenting achievement of student learning objectives, business schools had some difficulty in using assessment data to improve curricula (Martell, 2007). The new 2013 AACSB standards placed even more emphasis on the importance of linking assessment data to curriculum management (AACSB, 2013).
Research Purpose and Research Questions
The purpose of this study was to document the extent to which AACSB-accredited business schools have imple-mented strategies to improve students’ ability to achieve
program learning objectives. Other schools can learn possi-ble answers to the question, “How do we ‘close the loop’ and improve student achievement?” Specifically, we attempted to identify (a) purposes of assessment; (b) the competencies assessed; (c) assessment methods used; and (d) changes made in response to assessment. Business as a dynamic discipline potentially affects what competencies are emphasized and assessed and how schools are assessing them. Have business schools changed the learning objec-tives or assessment methods? Business school deans pay careful attention to these efforts and are especially qualified to provide related information. Therefore, the deans of AACSB-accredited business schools in the United States were selected as the population to provide relevant research data.
Literature Review
In previous research studies, business schools reported on the purpose of assessment, the competencies assessed, the types of assessment methods used, and the level of satisfac-tion with the assessment process. Few studies have addressed strategies for improving student achievement, although in more recent years, more data have been com-piled. According to previous research and case studies, business schools have certainly been engaged in the assess-ment process and used various ways to provide evidence of
Correspondence should be addressed to Barbara M. Wheeling, Mon-tana State University Billings, College of Business, 1500 University Drive, Billings, MT 59101, USA. E-mail: barbara.wheeling@msubillings.edu ISSN: 0883-2323 print / 1940-3356 online
DOI: 10.1080/08832323.2014.973824
students’ ability to meet program learning objectives. Trends suggest, however, that measured competencies dif-fer over time and difdif-ferent assessment methods are used. Those who have addressed strategies for improving student achievement indicate several different approaches can be employed. In other words, one size does not fit all.
Previous published work on assessment practices includes research studies of business schools and depart-ments within business schools (Calderon, Green, & Hark-ness, 2004; Hindi & Miller, 2000a, 2000b, 2007; Hindi, Najdawi, & Al Muftah, 2011; Kelley et al., 2010; Lusher, 2010; Martell, 2007; Pringle & Michel, 2007). These stud-ies explain the importance of assessment in continuous quality improvement and accountability. Business schools are tasked to present evidence on how well the school is meeting the expectations of the mission, its stakeholders, and accreditation (Hindi & Miller, 2000a). Early research studies of business deans (Hindi & Miller, 2000a, 2000b) reported the three most important purposes of assessment are monitoring the effectiveness of academic programs, planning and improvement efforts, and meeting accredita-tion agency requirements.
Business schools assess a variety of skills and competen-cies. Among the most-assessed competencies reported in early research studies (Hindi & Miller, 2000a, 2000b) are communication skills, professional knowledge, critical thinking, and problem solving. Other competencies men-tioned were technology–computer usage and interpersonal skills. The least-assessed skills reported in these research studies were knowledge and comprehension, global issues, professional integrity and ethics, and lifelong learning. A follow-up survey of accounting departments (Hindi & Miller, 2007) reported little change in the most-assessed skills and competencies. Communication skills, profes-sional knowledge, critical thinking and problem solving ranked at the top of the list of skills and competencies assessed. Global issues, professional integrity an ethics, and lifelong learning, along with reflective thinking and multicultural and diversity issues ranked as the least-assessed skills and competencies.
Assessment methods have evolved from an emphasis on standardized tests (direct assessment) and research studies of students, faculty, employers, and alumni (indirect assess-ment) to more emphasis on direct assessment methods. Hindi and Miller (2000a, 2000b) reported only 24% and 17.7% of participants, respectively, indicated the use of assessment methods other than standardized tests and ques-tionnaires. By 2006, well over half of business school par-ticipants reported greater use of written, oral, and course-embedded assignments graded with rubrics, with only 46% of schools reporting use of the Educational Testing Service Major Field Test (Martell, 2007). Other direct assessments reported include evaluation of teamwork, simulations, and written business plans. The least-reported assessment meth-ods were mock interviews, common school exams,
psychometric measures, and assessment centers. Pringle and Michel (2007) reported similar results. A comparison of their research data with Martell and Calderon (2005) revealed less use of questionnaires administered to graduat-ing students, alumni and employers. Similarly, Prgraduat-ingle and Michel reported a high percentage of participants (71%) using course-embedded activities.
Business schools have reported a number of strengths and weaknesses of the assessment process (Hindi & Miller, 2000a). Suggested areas to improve or revise include a greater number of assessment measures, increased objectiv-ity and improved learning outcome measures. Hindi and Miller’s (2000b) survey of accounting department chairs reported that only 47% of participants were satisfied with their assessment processes, and 20.7% were dissatisfied. Of the participants, 63.4% planned to improve their assessment processes through greater focus on assessment, continuous review of the assessment process, more feedback from employers and alumni, and more assessment measures.
In the early years of assessment, a need for improvement in the assessment process is expected. Over time, as these improvements occur, new areas for improvement are likely to emerge. Among the purposes of assessment, improving student learning is particularly important to colleges and universities. After data are collected and summarized, fac-ulty members consider where students are failing and how student learning can be improved. Pringle and Michel’s (2007) survey indicated most business schools made minor changes to the curriculum (96% of participants), although specific changes were not mentioned. More than half the participants identified modifications to student learning out-comes (76%), modifications to teaching styles (58%), and closer coordination of multi-section courses (53%). To a lesser extent, participants identified major changes to the curriculum (32%) and modifications to grading methods (22%) as ways to improve student learning.
Recent research identifying ways to improve student learning based on assessment results includes Kelley et al.’ s (2010) study. More than half the participants indicated modifications to the required core curriculum (63%), stu-dent learning outcomes (57%), and teaching styles as the top three changes to improve student learning. Closer coor-dination of multisection courses was also indicated (43%). Modifications to teaching methods were reported by only 29% of participants, and major changes to the curriculum were reported by only 20% of participants. Modified grad-ing methods were reported by only 8% of participants. These percentages are lower than those reported by Pringle and Michel (2007); however, this study also reported greater use of out-of-classroom experiences and new admission standards (14% and 8% of participants, respectively).
Although research studies can provide useful informa-tion on strategies to improve students’ achievement of learning objectives, case studies of departments or schools
ASSESSMENT AT AACSB SCHOOLS 45
may reveal more specific strategies that might be more helpful to anyone struggling with what to do. Kennesaw State University (Stivers, 2000) made changes to course content by requiring oral and written assignments in all upper level accounting classes to improve communication. It also put a greater emphasis on critical thinking in course assignments by requiring students to explain their conclu-sions in the analysis of a situation, and it changed the cur-riculum by adding prerequisites and changing some courses from electives to required courses. Changes to required courses were also reported by the Virginia Military Institute (Bush, 2008), which also made faculty changes and policy changes to provide higher quality student learning. Changes to course content and student assignments were also reported in Ammons (2005), Hollister (2007), and Law-rence (2011). Another study reported changes in instruction and methods of providing feedback to students to improve motivation (Ammons, 2005).
The following section reports the research process used to survey deans of AACSB-accredited business schools in the United States regarding strategies to improve students’ ability to achieve program learning objectives.
METHOD
Instrumentation
To assemble a comprehensive view of the current state of assessment, a questionnaire was developed that included the purposes of assessment, the skills and competencies assessed, the assessment instruments in use, whether indi-vidual or group performance is assessed, and changes made as a result of assessment. The articles cited in the preceding literature review were used to indicate valid and appropriate questions. Before the questionnaire was used, three profes-sors with many years of assessment experience examined the questions with consideration given to both the validity (relevance and appropriateness) and the reliability (unam-biguous meaning) of the questions.
Research Procedure
AACSB-accredited schools need to be sensitive to chang-ing accreditation requirements and the utility of assessment in satisfying those requirements, and business school deans pay careful attention to their schools’ efforts to improve assessment; thus the deans of AACSB-accredited business schools were chosen as an appropriate population to pro-vide information about the current status of assessment in business schools. Therefore, the questionnaire was mailed to the 491 deans of AACSB-accredited business schools in the United States. A follow-up questionnaire was not used. Because the intent was to develop an understanding of the current state of assessment in accredited schools, data
analysis was limited to reporting the percentages of schools responding to the questions in various ways.
RESULTS
Participant Data
Responses were received from 91 schools, a response rate of 18.5%. Table 1 indicates a wide range of schools (a total of 91 schools) was represented, with approximately 25% (23 schools) having over 2,000 students, roughly 40% (35 schools) having between 1,000 and 2,000 students, and about 35% (33 schools) having fewer than 1,000 students.
Because the results are of interest to many administrators and faculty members, a detailed summary is provided in tables. The following paragraphs focus on highlights of responses.
Purposes of Assessment
Regarding the purposes of assessment, Table 2 presents the numbers of participants who indicated various purposes were relevant. When asked to indicate the purposes of assessment at their business schools, 82 deans (slightly more than 90%) indicated assessment was used to meet accreditation requirements, and another 83 participants stated assessment was used to guide planning and improve-ment efforts. Monitoring the effectiveness of programs was a purpose indicated by 82 participants (90%). On the other hand, only nine deans considered attraction of equipment
TABLE 1
Total Students (Full- and Part-Time) in Participants’ Programs
Students n % of schools 3,000 or more 11 12.1 2,001–3,000 12 13.2 1,001–2,000 35 38.4 501–1,000 19 20.9
201–500 13 14.3
200 or fewer 1 1.1
Total 91 100
TABLE 2 Purposes of Assessment
Purpose n % of schools Guide planning and improvement efforts 83 91.2 Meet accreditation agency requirements 82 90.1 Monitor the effectiveness of the programs 82 90.1 Increase accountability 55 60.4 Provide information relevant to policies 23 25.3 Attract better students 19 20.9 Attract equipment or financial resources 9 9.9
or financial resources to be a purpose. These results are con-sistent with the earlier research mentioned.
Competencies Assessed
Table 3 summarizes skills or competencies currently assessed at the various business schools. As with earlier research studies, communication was the most-assessed skill, as reported by 87 deans (96%). This skill was closely followed by professional business knowledge, critical thinking, and professional integrity and ethics, all reported by over 80% of participants. Of these, professional integrity and ethics was reported by more schools than in previous research, suggesting greater emphasis of this topic in busi-ness curricula. In terms of frequent practices, roughly three fourths of the schools assessed global issues and problem-solving skills. Global issues was previously one of the least assessed skills. Slightly more than a third of the deans (33 schools) reported assessing multicultural and diversity issues (previously one of the least assessed skills), and only two participants (2.4%) noted assessment of lifelong learn-ing issues.
Assessment Methods Used
As shown in Table 4, participants had an opportunity to note assessment methods presently being utilized. Consis-tent with the trend towards direct assessment methods, 81 deans (89%) reported usage of rubric-scored written assign-ments. More than 80% of participants (76 schools) used test items embedded in exams and classroom activities or rubric-scored oral presentations. While somewhat more than half the deans (52 schools) reported usage of standard-ized exams from outside vendors, in-house designed exams, and rubric-scored cases, nearly half (40 schools) used writ-ten questionnaires administered to graduating students. Almost a third (28 schools) used student evaluations of fac-ulty or research questionnaires administered to alumni. The least-used methods—used by fewer than 3% of the business
schools—included case studies not part of a specific class (two schools), questionnaires administered to faculty (one school), role-playing interviews (two schools), and formal assessment centers (two schools). These results are consis-tent with recent trends in assessment methods.
TABLE 3
Skills and Competencies Assessed
Skill or competency n % of schools Communication skills 87 95.6 Professional business knowledge 81 89.0 Professional integrity and ethics 78 85.7 Critical thinking 77 84.6 Problem-solving skills 69 75.8 Global issues 67 73.6 Technology and computer skills 48 52.7 Interpersonal skills 40 44.0 Multicultural and diversity issues 33 36.3 Reflective thinking 18 19.8 Sustainable business practices 12 13.2 Lifelong learning 2 2.2
TABLE 4 Assessment Methods Used
Method n % of schools Rubric-scored written assignments 81 89.0 Written assignments that are embedded in
classroom activities
79 86.8 Rubric-scored oral presentations 77 84.6 Test items embedded in regular course exams
or final exams
76 83.5 Standardized exams from outside vendors 52 57.1 Rubric-scored cases 51 56.0 Exams designed in house 48 52.7 Written survey of graduating students 40 44.0 Pre- and posttests of written or behavioral
measures
30 33.0 Survey of alumni 29 31.9 Student evaluation of faculty 28 30.8 Job placement of graduates 27 29.7 Class assignment involving computer
simulation
24 26.4 Exit interviews with individual students who
are graduating
22 24.2 Evaluations of student performance by outside
experts
20 22.0
Students’ overall performance in a capstone class
20 22.0 Supervisor evaluations of student interns 20 22.0 Survey of employers of alumni 19 20.9 Other rubric-scored course embedded
assignments
17 18.7 Questionnaires measuring attitudes, beliefs,
personality traits, etc.
17 18.7
Performance on licensing exams 16 17.6 Evaluations of student performance by a
faculty panel
15 16.5 Behavioral deliverables that are embedded in
classroom activities
10 11.0 Admission rates of graduates to graduate
schools
10 11.0 Student competitions 9 9.9 Evaluations of student portfolios 8 8.8 Oral presentations that are not part of a
specific class
6 6.6 Focus groups of graduating students 6 6.6 Business plan written by single student (not a
team project)
3 3.3 Role playing exercises that are not part of a
specific class
3 3.3
Videotaped exercises that are not part of a specific class
3 3.3 Case studies that are not part of a specific class 2 2.2 Role-playing interviews 2 2.2 Formal assessment center 2 2.2 Faculty survey 1 1.1
ASSESSMENT AT AACSB SCHOOLS 47
Changes Made in Response to Assessment
To learn about uses of assessment data, deans were asked to report changes initiated as a result of assessment practices. As shown in Table 5, 72 deans (79%) indicated mandated increased coverage of specific skills or topics across the curriculum, which was by far the greatest usage of the data. In previous research studies, minor modifications to the cur-riculum was the change most frequently reported. Modifica-tion of student-learning objectives was noted by 59 participants (65%), and nearly half (43 schools) of those surveyed stated addition of required course(s) to the core or major curriculum. Previously, a major change to the curric-ulum was one of the least reported strategies. The fewest uses of data involved using mandated review tests prior to registration in a specific course (eight schools) and changes to composition of faculty (eight schools). Consistent with previous research studies, modification of grading methods (eight schools) was also one of the fewest uses of assess-ment data.
DISCUSSION
Assessment has received increasing interest in the field of higher education. The present research found the purposes of assessment to be similar to the purposes mentioned in earlier studies, and there is a continuing trend toward increasing the use of direct assessment methods.
Seemingly, several factors have contributed to this trend, including public concerns related to expenditures for higher education, standards established by accreditation organiza-tions, and increasing competitiveness among colleges and universities. Educators are challenged to prepare students for an environment characterized by increasing uncertain-ties and consistent emphasis upon productivity. Indeed, many responses from those surveyed corresponded with these likely expectations.
The deans of AACSB-accredited business schools were certainly aware of the need to comply with accreditation standards. In terms of integrating information into program enhancements, they recognized the importance of providing viable curriculum offerings and retaining a focus on continu-ous improvement as well as the need to close the loop. Earlier studies found that significant changes to the curriculum were implemented relatively infrequently. The present research found a large majority of schools are mandating increase cov-erage of specific skills or topics across the curriculum and add-ing required courses. As might be anticipated, communication skills were the most assessed skill or competency. Rubric-scored written assignments, test items embedded in exams, and classroom activities or rubric-scored oral presentations were prominent assessment methods.
Assessment requirements continue to evolve, and busi-ness schools need to stay in touch with these changes and continue to develop both their assessment methods and their responsiveness to assessment. The present study pro-vides a useful picture of current assessment activities and the use of assessment data to improve programs. The results indicate more business schools are assessing professional integrity and ethics, global issues, and multicultural and diversity issues than previously reported. Changes to the curriculum, including adding courses or prerequisites, is a more prominent strategy for improving student achieve-ment of learning objectives than previously reported. Future studies are needed to track these dynamic and crucial ele-ments in accredited business schools.
REFERENCES
Association to Advance Collegiate Schools of Business (AACSB). (2013).
Eligibility procedures and accreditation standards for business accredi-tation, adopted April 8, 2013. Retrieved from http://www.aacsb. edu/~/media/AACSB/Docs/Accreditation/Standards/2013-business-standards.ashx
Ammons, J. L. (2005). Course-embedded assessments for evaluating cross functional integration and improving the teaching learning process.
Issues in Accounting Education,20, 1–19.
Bush, H. F. (2008). Using the major field test in business as an assessment tool and impetus for program improvement: Fifteen years of experience at Virginia Military Institute.Journal of College Teaching and Learn-ing,5, 75–88.
Calderon, T. G., Green, B. P., & Harkness, M. D. (2004). Identifying best practices in accounting program assessment. In T. G. Calderon, B. P. Green, & M. D. Harkness (Eds.),Best Practices in Accounting Program Assessment(pp. 17–31). Sarasota, FL: American Accounting Association.
TABLE 5
Changes Made as a Result of Assessment
Change n % of schools Mandated increased coverage of
specific skills or topics across the curriculum
72 79.1
Modifications to student learning objectives
59 64.8 Modifications to teaching methods or
styles
52 57.1 Addition of required course(s) to the
core or major curriculum
43 47.3 Modifications to prerequisites 38 41.8 Faculty development for teaching
specific skills or topics
31 34.1 Appointed coordinators for
multiple-section courses to ensure consistency
27 29.7
Increased use of out-of-the classroom learning experiences (e.g., internships, service learning)
26 28.6
Mandated review tests prior to registration in specific courses, with remediation provided for failures
8 8.8
Modifications to grading methods 8 8.8 Changes to the composition of the
faculty
8 8.8
Hindi, N. M., & Miller, D. (2000a). A survey of assessment practices in schools of business.Central Business Review,19(1), 13–18.
Hindi, N. M., & Miller, D. (2000b). A survey of assessment practices in accounting departments of colleges and universities.Journal of Educa-tion for Business,75, 286–290.
Hindi, N. M., & Miller, D. S. (2007). Assessment practices in accounting departments of U.S. colleges and universities.Studies in Business and Economics,13, 51–67.
Hindi, N. M., Najdawi, M. K., & Al Muftah, H. A. (2011). An examination of assessment practices in colleges of business at various Middle East countries compared to the USA. International Journal Management Development,1, 40–59.
Hollister, K. K. (2007, October).Curricular changes in response to assur-ance of learning results in information technology. Paper presented at the IABE-2007 Annual Conference, Las Vegas, NV.
Kelley, C., Tong, P., & Choi, B.-J. (2010). A review of assessment of stu-dent learning programs at AACSB schools: A dean’s perspective. Jour-nal of Education for Business,85, 299–306.
Lawrence, K. E. (2011). Experiencing and measuring the unteachable: Achieving AACSB learning assurance requirements in business ethics.
Journal of Education for Business,86, 92–99.
Lusher, A. L. (2010). Assessment practices in undergraduate accounting programs.Journal of Case Studies in Accreditation and Assessment,1, 1–20.
Martell, K. (2007). Assessing student learning: Are business schools mak-ing the grade?Journal of Education for Business,84, 189–195. Martell, K., & Calderon, T. G. (2005). Assessment in business schools:
What it is, where we are, and where we need to go now. In K. Martell, & T. G. Calderon (Eds.),Assessment of has become student learning in business schools: Best practices each step of the way(Vol. 1, pp. 1–26). Tallahassee, FL: Association for Institutional Research.
Pringle, C., & Michel, M. (2007). Assessment practices in AACSB-accred-ited business schools.Journal of Education for Business,82, 202–211. Stivers, B. P. (2000). An assessment program for accounting: Design,
implementation and reflection. Issues in Accounting Education, 15, 553–581.
ASSESSMENT AT AACSB SCHOOLS 49
