Too often, in the words of Parker Palmer, we shut down and engage in assessment processes that seem inauthentic, beside the point, or even undermine our work. And all of that has led us to share this document with our fellow Imagining America (IA) members and our colleagues on campuses and in broader communities who work to assess community engagement and public scholarship. The appendices to the white paper include five examples of our detailed critique of sample assessment tools through the lens of DEA (Appendix A) and a set of tools to support assessment in multiple community engagement contexts (Appendix B).
Our use of the words "democracy" and "justice" also warrants qualification in an attempt to avoid any frustration or confusion.
Conceptualizing Democratically Engaged Assessment
Mary's story highlights some of the complications facing attempts to cultivate DEA in this light. Generativity can manifest across any of the three key domains of assessment: process, relationships, and outcomes (see Patti's assessment story provides an example of the development, implementation, and refinement of an assessment process—in this case, focused on student learning—that we can investigate with that.
The dashed line in the figure represents an imagined assessment project and the relative tensile strength of the values in relation to each other as we do assessment.
Putting DEA into Practice
How will the perspectives of the entire range of stakeholders be taken into account when setting values and goals, determining audiences and choosing methods. Who is and who is not involved in the implementation and interpretation of the created information. Does the entire range of participants feel empowered to contribute at each stage of the evaluation?
Given equal value to the knowledge, questions and abilities of the full range of participants in the co-creation of the research design and methods. Is the whole range of participants who contribute to the implementation of methods and the creation of meaning of the results. Allows the tool to involve the full range of participants across all phases of the assessment.
The tool effectively utilizes the assets and capabilities of the full range of participants. On what grounds, and to what extent, will the instrument ensure accuracy and reliability [validity] of the results. Support the use of the tool reflective criticism which can yield growth and development of people, organizations and processes.
Will the use of the tool enable social and ecological resilience in our organizations, communities or society. Will the use of the tool contribute to lasting capacity and meaningful learning for the participants.
Tensions and Tension-Holding in DEA
Figure 3 provides a representation of some of the tensions we see when trying to reimagine and implement them in challenging contexts. The situations we encounter in the world, and in light of the values we value, can easily trigger tensions within us, both as individuals and to varying degrees collectively. What is the appropriate role of the 'expert' in assessment in light of these competing views of the relationship between credentialed professionals and other partners.
How do we understand the role of experts in assessment and in light of the DEA values of full participation, co-creativity, generativity, rigor,. Partners redrew maps in phases with each iteration providing a more inclusive, while still situated, view of the cross-cutting relationships in the partnership. It is important to first recognize the closest cousin of the "shutdown" or compliance strategy: the exit strategy.
In the Street Theater story we examined in Parts I and II, the theater company followed a Story Circle evaluation method despite disinterest from the funding university. The rest of the story reveals that the theater company's insistence on doing the review "their way" eventually led to it. Now we know what you're talking about!” It was unanimous in the room that receiving this part as part of the grant made their work, their research and their statistics meaningful.
In the case of the liberal arts college, the center's staff decided to decline involvement with the assessment specialist rather than invite further collaboration with her and other decision makers on and off campus. However, the importance of these capabilities does not fade over the course of the project. Again, an exhaustive review of the vast literature on social movements and social change is beyond the scope of this white paper.
For many in the community engagement movement, this transformation of the academy is but part of a broader transformation of the relationship between higher education and society as a whole.
Questions for Further Inquiry
How can DEA be conceived, understood and applied with a greater sense of full participation and co-creation? Our voices are also positioned in other ways that we must name and interrogate if we are to maintain our integrity and question our limitations. This leads us to ask: How might DEA be created and communicated if our group included broader representation across lines of race, ethnicity, gender, sexuality, age, ideology, education, and national origin, to name a few? to name. We view living these values more fully as an ongoing commitment and would like to pursue more deeply participatory and co-creative work in the future development of DEA, including building cases, developing contextual applications, and reformulating the principles of DEA.
What DEA will be in the future as we more fully implement our commitments to its values and as applied by ourselves and others shaped by a wide variety of experiences. This raises the question: How can DEA be challenged and expanded to address pressing issues at the scales and timescales that larger groups, institutions, and communities need to drive. How would the values of DEA conflict or require compromise in realizing the assessment at this macro-level societal scale.
Are there ways in which these can be harnessed, particularly in the humanities, arts and design, to develop creative tools that extend or challenge DEA. How might the DEA be challenged by contexts outside the United States? We, as a group of authors, are all from the United States, which other hierarchies of power in the assessment will come into focus between, for example, colonial and anti-colonial methods.
How can DEA be further developed through the use of different methods and tools? different methods and in different evaluation contexts. It is our hope and intention that the continued development of DEA will continue in the spirit of - and contribute to - its flourishing.
Conclusion
Our tools and practices also assert that DEA cannot—and should not try—to proceed in hopes of avoiding tension. Such tensions, once recognized and accepted, allow us to dance more powerfully and smoothly, and in doing so, create beautiful new possibilities. Just as Socrates thought it necessary to create a tension in the mind so that individuals could rise from the slavery of myths and half-truths into the unfettered realm of creative analysis and objective evaluation, we must see the need for flies not violent to create the kind of tension in society that will help men [sic] rise from the dark depths of prejudice and racism to the majestic heights of understanding and brotherhood [and sisterhood].
While tension is part of the dance, part of the change process, we find that articulating, practicing, and defending DEA becomes easier as our cultures see more of it, as both the intellectual framing and its practical application become more usual.
Appendix A
DEA Analysis of Tools & Methods
Are we where we want to be in terms of the quality of our partnership. Reliable implementation of the instrument as a component of a more broadly focused investigation of partnership quality (as described above) would improve rigor. Representatives from various types of institutions piloted the first version of the tool and provided feedback to refine it before the classification was launched.
Continuous refinement of the tool for each cycle draws from input from campus applications (the question asks for such input) as well as practitioner-scholars in a. The tool provides information that can lead to change and growth at all levels of the institution and in terms of all three areas: process, relationships and results. The analysis of the Carnegie Elective Community Engagement Classification in the context of the assessment phases is presented in II. work.
The following items taken from the scale illustrate the competencies addressed by this approach. Knowledge of the role of community-engaged scholarships in my institution's review, promotion, and tenure process. The scale is flexible in that it is possible to focus on a subset of items; the.
Students, professional staff, and faculty all functioned as equals in the development and refinement of the tools. Although a user can certainly select a subset of critical thinking standards to focus on in any particular situation, the tools are not one-size-fits-all. The tools are flexible in what a subset of standards can focus on (a practice that some have found useful and others problematic).
The broad applicability of the instruments enables sustainable refinement of critical thinking skills in multiple situations (e.g., over a semester, through training, throughout life).
Works Cited
Service learning and social justice education: Strengthening justice-oriented community-based teaching and learning models. Paper presented at the 11th International Society for Research on Service Learning and Community Engagement. Session facilitated at the 11th Annual International Conference on Service-Learning Research and Community Engagement Conference, Chicago, IL.
Assessing service learning and civic engagement: principles and techniques (2nd ed.). Community-university partnerships for mutual learning. Paper presented at the 17th International Association of Service-Learning and Community Engagement Conference, Galway, Ireland. Paper presented at the 14th Annual International Association for Research on Service-Learning and Community Engagement Conference, New Orleans, LA.
