Civic engagement evaluation: An important and difficult task

Spencer Foundation president Michael McPherson has been an outspoken proponent of qualitative, rather than quantitative, civic engagement assessment. In an article for Campus Compact, he notes:

It is natural for those scholars and practitioners who are interested in the effectiveness of civic engagement programs in colleges to think about results in quantitative, “head count” terms. We talk about how many people vote, or participate in political parties, or join in community improvement campaigns. There is, I find, less attention to the quality of those engagements. I think this is unfortunate.”

At the end  of the article he concedes that he is not arguing for solely qualitative analysis, and that there is room for both. His arguments are part of a larger conversation about civic engagement and how to best assess our efforts and outcomes.

The need for assessment is clear. Like many universities, Duke has a decentralized landscape of programs and offices. We also (from our unbiased standpoint!) have a spectacularly motivated population of students, faculty, and staff. This population is inventive, entrepreneurial, and smart. This all means: Duke has a plethora of programs and initiatives. There are programs large and small, run by students, by faculty, by staff, and even by our alumni. New programs are constantly created, while others may stop running as students graduate or institutional priorities and resources change. This shifting landscape makes it challenging to get a bird’s eye view of the many opportunities to engage through Duke.

For those involved in civic engagement through Duke, it would be very useful to have a picture of all the ways our campus engages with local, national and international communities.  It is often amazingly difficult to definitively answer any question that begins, “Do you know who at Duke is working on ___?”  or, “Who else is working with ___ organization?”   Assessment is important for many purposes. Accreditation agencies want to be able to evaluate how Duke is civically engaged and how its engagement compares to other schools’.  In order to gain national recognition from entities such as the Carnegie Foundation’s Community Engagement Classification and the President’s Higher Education Community Service Honor Roll, Duke needs to be able to present data on how, and how much, our university is civically engaged.

Nationally, the demand for data-driven assessment is growing. From Moneyball-style sports analyses to stats-driven politics in the style of Nate Silver, quantitative data has a huge market. In higher education, universities are judged (fairly or unfairly) by data such as acceptance rates, average SAT score, and student tests such as the Collegiate Learning Assessment.

The Challenges of Assessment

In civic engagement, we are not talking about “big data,” analyzing huge datasets to find overall trends and essentially predict the future; we’re simply trying to find and analyze trends in how our campus communities engage with outside communities. Yet it’s still a tricky proposition.

McPherson questions the relevance of quantitative data as an effective measure of a University’s civic engagement. Nationally, many people are asking tough questions:  Is it more important to have a large number of students involved in civic engagement, or to have those who are involved be more deeply engaged? What can we learn by counting the number of service hours performed? What are the limitations of such a count? Does it matter whether university participants benefit from their civic engagement, or is the community benefit what matters most? How does one measure advocacy efforts? In social innovation, does outcome matter more than process? How do we measure the extent to which civic engagement is integrated with research and teaching?  Our answers to these questions are highly subjective and depend on the values and viewpoints of the person asking.

The DOCE’s Role

Since our office’s inception, we’ve worked to gather data. We have identified five ways Duke engages: service, advocacy, social innovation, engaged research, and community development. We’ve created an exhaustive list of courses and student activities that engage in four of these activities, and have gathered data on university programs and initiatives in all five areas. We are also in the process of creating a list of faculty whose research involves activities in one of these areas, with help from the new Scholars at Duke database.

This fall we are beginning a project, in collaboration with the Duke-Durham Neighborhood Partnership, to identify all of Duke’s community partners in Durham, a process that will, in all likelihood, be onerous. We are also working to distribute a survey developed in collaboration with members of the UCCE (University Council on Civic Engagement) to identify the priorities and impact of community service programs on our campus. We plan to combine much of this data in an online annual report intended to give a snapshot of all of the various ways we are engaging through Duke.

We are endeavoring this data collection while acknowledging fully that quantitative data has limitations. We also believe that it’s an important part of the broader picture of civic engagement at Duke. We are watching and participating in national conversations about evaluating and assessing civic engagement, and as always, we exist as a resource to the whole campus. The data we collect is only useful if it’s data the Duke community wants and needs. We welcome your input and your feedback.