Deck 4: Cultural Sensitivity and Responsiveness in Evaluation

Question 1

Evaluate the impact of the American Evaluation Association (AEA) Public Statement on Cultural Competence in Evaluation.

Answer 1

The Statement is grounded in and goes beyond the 2004 American Evaluation Association Guiding Principles for Evaluators, which guide ethical practice, to explore cultural competence and its significance in evaluation. It was crafted by the Cultural Competence in Evaluation Task Force of AEA's Diversity Committee and approved by the AEA membership in 2011. The Statement is important because it (1) affirms the importance of cultural competence as a criterion for assessing the quality of evaluation efforts, (2) indicates that the public should pay attention to cultural competence, (3) confirms the importance of practicing evaluators attending to relevant dimensions of culture, and (4) illuminates essential elements of cultural competence that are relevant to the interpretation and use of evaluation findings. The Statement established that addressing culture, context, and common good are integral to ethical evaluation practice, the production of high-quality evaluations, and the use of evaluation findings.

Question 2

After reading the material in the chapter, discuss the role of culture and cultural competence in quality evaluation.

Answer 2

The Statement defines culture as "the shared experiences of people, including their language, values, customs, beliefs, and mores." In this way, the Statement challenges the traditional belief that evaluation is an objective, culture-free enterprise. Instead, it asserts that evaluation is culture-bound and values-based. Culture and the associated values of evaluators and of those sponsoring an evaluation influence every phase and all aspects of an evaluation, including what is being evaluated, how it is being studied, what data are collected, how the data will be analyzed and interpreted, and which groups are included or excluded from an evaluation process. Furthermore, these cultural values are used to make judgments about the merit or worth of what is being evaluated, whether one recognizes it or not.
According to the Statement, "cultural competence is not a state at which one arrives; rather, it is a process of learning, unlearning, and relearning." Cultural competence is defined in relation to a specific context or location, such as geography (e.g., the Pacific Northwest vs. Central Texas), nationality (the place or country of citizenship or birth or the place from which a person descends shapes values, customs, beliefs), and history (e.g., past experiences such as abuses of groups of people influence their patterns of behavior such as adaptive cultural mistrust and parenting practices transmitted from one generation to the next). Thus, for each evaluation, a culturally competent evaluator (or evaluation team) must have specific knowledge of the people and place in which the evaluation is being conducted, including local history and culturally determined mores, values, and ways of knowing. The Statement indicates that cultural competence is effectively a stance toward culture. For example, evaluators must not assume that they fully understand the perspectives of stakeholders whose backgrounds differ from their own; they must make a concerted effort to understand and reflect those perspectives in the work.

Question 3

List and discuss the four essential practices for developing and implementing a culturally competent evaluation.

Answer 3

The practices that are essential for developing and implementing a culturally competent evaluation include (1) acknowledge the complexity of cultural identity (2) recognize the dynamics of power, (3) recognize and eliminate bias in language, and (4) employ culturally appropriate methods.
(1) Acknowledge the complexity of cultural identity: Although culture is often conflated with race and ethnicity, it refers to the sum of social identities (i.e., the categories which people are socialized to classify themselves and others) including age, ability status, class, gender, gender identity, political affiliation, religion, and sexual orientation. Based on the set of identities to which a person subscribes (or is prescribed by others), they may view themselves as composed of multiple identities that are simultaneously operating, or they may experience conflicts in how they negotiate these multiple identities because of incompatible messages they receive from prevailing societal norms, cultural stereotypes, and self-perceptions. To this end, cultural identity is multidimensional and should not be conceptualized through a single lens. Layers of cultural differences exist within groups, such as skin tone variation within ethnic minority groups; different clans and clan systems within a single Native American tribe; and differences in political perspectives, acculturation, religion, generational perspectives, and educational background within a single nationality. It is incumbent upon evaluators to be aware of and to elucidate these complexities. Evaluators must recognize that cultural identity is multifaceted and can defy precise categorization. The evaluator must consider the multiple identities that evaluation participants may have, including a recognition of issues of intersectionality (i.e., multiple dimensions of identity; potential inequities may be the result of multiple social locations, power dynamics, and experiences), diversity within and between groups, and the right of individuals to define their own cultural identities (e.g., Latino/Latina, Latinx, Hispanic, Chicano, or identification by nationality, etc.).
(2) Recognize the dynamics of power: Evaluators should be aware of how their own cultural identities impact how they understand the world and how they approach the evaluation process. Evaluators should recognize the presence and dynamics of power (e.g., who holds power and how that power is used to influence the behavior of others) and privilege (advantages based on social identity) in all human interactions as well as the inferior social status and subordination that people experience based on their cultural identities. Evaluators should also recognize how these dynamics influence evaluation (e.g., inclusion or exclusion of certain stakeholders/participants in the evaluation process, decision-making authority, evaluators' own position of privilege and power within the evaluation). Because evaluation data and findings have the potential to contribute to structural change, evaluators should recognize the power of evaluation to illuminate experiences of marginalization and subjugation among socially "othered" groups and the subsequent role of evaluation for promoting equity and inclusion.
(3) Recognize and eliminate bias in language: Because language is both intentionally and inadvertently used to denigrate oppressed groups, evaluators must recognize the critical use of language as a means for promoting social justice and must avoid using language that perpetuates stereotypes, bias, and social exclusion (e.g., "at-risk youth" vs. "youth at risk for poor developmental outcomes"). When used thoughtfully and purposely, language can contribute to a culturally competent evaluation by (a) respectfully communicating differences in key stakeholders' worldviews, (b) illuminating institutional bias and stereotypical practices, (c) capturing the nuance of participants' cultural identities in lieu of oversimplified social classifications, and (d) integrating participants' perspectives in their preferred language or mode(s) of communication.
(4) Employ culturally appropriate methods: Evaluators should recognize that the common instruments and methodologies used in evaluation for data collection, analysis, and reporting are only optimally effective for the populations for which they were developed or normed. To use culturally appropriate methods, evaluators must (a) define constructs of interest based on how the groups being evaluated define, interpret, and understand them; (b) determine culturally informed methods to analyze/present findings and use or develop culturally validated measures; (c) engage in regular reflection on the ethical collection and use of data; (d) employ intermediaries for data collection in cases in which valuable perspectives may be lost due to barriers in language, abilities, or trust; (e) generate multiple reports that communicate findings at varying levels of comprehension; and (f) incorporate community feedback in the meta-evaluation process as a way to continue building cultural competence.