Philanthropic Evaluation: Toward an Antiracist Future of Practice

Introduction

This study examines a recent upward trend in the number of foundations embracing racial equity as a core value. It identifies  philanthropic evaluation, a practice that has yet to embrace this core value. The author, Jara Dean-Coffey, explores how race and racism have affected philanthropic evaluation practices and contends that these practices may inadvertently reinforce racism. The author addresses this problem by tracing the co-evolution of both disciplines, identifying its racist myopia, and offering a framework for equitable evaluation practices moving forward.

Jara Dean-Coffey is an evaluation practitioner who has worked with philanthropic foundations for over two decades to strengthen their evaluation skills. She is affiliated with the Luminare Group, a firm that consults with philanthropic organizations  to promote equitable interventions. 

Methods and Findings

This paper analyzes the formation of the evaluation field through a historical lens. The exercise of conducting evaluations  emerged from the federal government and academic research institutions to determine the allocation of public funds and monitor the effectiveness of those funds. From its inception, evaluation, as a practice, was not equipped to interrogate structural inequalities and formulate solutions that promoted equity.  The discipline was conceived by a select number of high-earning, white male industrialists who developed philanthropic foundations (Ford, Rockefeller, Sage) and used evaluations to determine outputs and costs for their foundation work. In the 1980s, the federal government expanded its internal evaluation practices, leaving many evaluation professionals searching for new clients, namely philanthropic foundations. These foundations were identified as an industry through which public dollars could be leveraged to implement large-scale programs with quantifiable, objective data and rigid scientific rigor that needed oversight. In this setting, context and culture were assumed as control variables instead of critical elements of the program design and evaluation approach, which renders equity practices extremely difficult to measure. 

The author then deconstructs the evaluation discipline as a mode of defining, describing, and analyzing the world from the vantage point of a narrow set of definitions, descriptions, and analyses. The racialized distribution of public goods and services has historically favored white “publics” while severely undeserving non-white individuals. According to Dean-Coffey, the consequence of this historical omission is a framing effect that  relies on entrenched assumptions of what is normal and perhaps even right. This unconscious process has undermined the imagined possibilities of alternate evaluation practices that aim to decouple from their historically racist biases. As the nation has been forced to reckon with its structurally embedded racism, the evaluation discipline has recently attempted to generate philanthropic funds based on data showing racialized inequalities. Yet, these efforts have largely reified the prevailing limiting assumptions that advance the scientific logic of its founders. 

In response, Dean-Coffey sets forth recommendations for more equitable evaluation frameworks:

Equitable Evaluation Principles

Conclusions

Foundations aiming to promote equitable evaluation must first acknowledge the privileged and racist paradigms that cloak evaluation as objective and neutral. They must explore their personal values, principles, and orthodoxies that generate barriers to equitable evaluation and generate new practices to interrupt these patterns. This includes engaging in a new methodological approach, which tests for variables like culture and context to deconstruct the prevailing assumptions about knowledge, truth, and evidence. Evaluators must become more diverse, and their training must consider multiple modes of conceiving knowledge to produce more robust validity and rigor. 

Organizational behavioral modification should be used to make social justice a professional norm

Introduction

The professional field of evaluation plays an integral role in creating the results, measurements, and evidence that guide the worldviews and decisions of businesses, educational institutions, philanthropies, public health agencies, and governments. Given this influence, it is paramount for evaluators to understand the practice of evaluation as a tool to reinforce and enable or conversely to challenge the unequal, inequitable access and outcomes that minoritized communities experience. For example, evaluative standards have been manipulated at various points in an attempt to validate Black intellectual differences as evidence of deficiency.

Like other social science fields, the profession of evaluation must reckon with the ways that its norms, history, and practices are built upon and perpetuate structural racism and injustice. In this study, the authors argue that the field of evaluation cannot be reformed without addressing the structural racism embedded throughout. Instead, the field must not only identify and acknowledge its own structural racism, but also take concrete action to eradicate ingrained structural racism from its theories and practices.

Dr. Leon D. Caldwell, PhD, is the Senior Director of Health Equity Strategy and Innovations at the American Hospital Association and Founder and Managing Partner of Ujima Developers. Dr. Katrina L. Bledsoe, PhD, is a Senior Evaluation Specialist and Research Scientist at the Education Development Center, Principal Consultant at Katrina Bledsoe Consultancy, and Partner at Strategic Learning Partners for Innovation.

Methods and Findings

The authors first review the history of the field of evaluation’s struggle to reckon with both its partiality and role as an intentional political tool for maintaining Eurocentric structures of power and authority. Because the field presents itself as the authority on measurement of various relevant, consequential outcomes, acknowledgement of this reality is necessary to understand how its approaches to hypothesis generation, definition of data, and interpretation of findings are inherently limited and insufficient. The authors point to publications and conference addresses dating from the late 1970s to late 1990s as seminal statements on the field’s complicity in structural racism. Although several authors have advanced more equitable evaluation methodologies since the 2000s, these methods are far from universally accepted best practice.

Next, the authors make distinctions between individual and systemic racism as significant to eradicating systemic racism from the field. Although individual evaluators should interrogate and address their internalized and interpersonal racism, they also have the power to collectively deconstruct systemic racism in the field by advocating for legacy organizations like the American Evaluation Association to establish and enforce social justice oriented norms. For example, the authors point to the potential normalization of frameworks such as Patton’s Seven Questions Concerning Race in the Field of Evaluation as an opportunity to move the field forward on eradicating systemic racism.

Ultimately, the authors leverage organizational behavioral modification, otherwise known as reinforcement theory, to make recommendations for lasting change. Organizational behavioral modification requires consistent consequences following antecedents to reinforce desired behavior and extinguish undesired behaviors. In this vein, the authors recommend incorporating social justice concerns into mandatory selection criteria for professional awards, publications, and promotions to positively reinforce social justice as an industry standard. Similarly, the authors recommend that the American Evaluation Association and individual funders make institutional accreditations, grants, and contracts contingent upon consideration and fulfillment of equity priorities as an additional tool.

Conclusions

In this article, the authors argue that members of the evaluation field must push the field’s institutions to recognize their histories of promoting and amplifying structural injustices through both their norms and pedagogies. Those institutions should then take steps to establish consequences to reinforce social justice-oriented transformation and to extinguish attitudes, behaviors, and outcomes that reinforce systemic oppression. 

The authors contend that evaluators not only participate within inherent structural racism of all US systems, but that they have the power to change both the perception and material outcomes of minoritized populations. Thus, the field must commit to deconstructing former best practices grounded in European traditions and reimagining the field with leadership from BIPOC evaluators. This review highlights the need for additional analysis and experimentation to determine social justice-oriented best practices and to measure their rates of implementation across the field.