Exam 17: Creating Collaborative Partnerships Through Participatory Evaluation to Shape Public Policy

Correct Answer:

Verified

Cousins's practical participatory theory of evaluation provides a conceptual framework for engaging diverse stakeholders to develop and implement an evaluation. A "participatory" approach to evaluation means that practitioners need to be directly involved in the planning and execution of the evaluation and, to that end, Cousins outlines three dimensions of that involvement. First, with respect to stakeholder selection, evaluators and their community partners need to decide who will be involved in the planning and execution of the project. Some participatory models mandate inclusion of a broad range of stakeholder groups (e.g., community members at-large), but practical participatory evaluation projects tend to limit involvement to program staff and other immediate users, as this will often (though not always) promote use of the evaluation findings. If, however, utilization will require the buy-in and involvement of more diverse groups, then the project should include more stakeholders. Second, with respect to the control of the evaluation process, practical participatory evaluation projects vary as to whether the evaluators or practitioners hold more control over the process. Cousins's theory does not stipulate whether one group should have more (or less) control, but rather emphasizes the importance of open communication about control of the project and finding ways to ensure participation from all groups. Finally, practical participatory evaluations vary in depth of participation, such that stakeholder involvement ranges from a consultant-type role (e.g., no decision-making responsibility) to deep participation (e.g., involvement in all aspects of the evaluation, including dissemination and utilization). In some projects, program staff might be quite hands-on and actually conduct much of the evaluation themselves and, in other contexts, it might be more useful for program staff to advise evaluators on these tasks and focus their attention elsewhere. Cousins's theory does not prescribe what mode of participation is "right"; instead, stakeholder engagement should be tailored to the specific project to promote local use of the evaluation findings.

Correct Answer:

Verified

The NIJ SAK Action Research Project evaluation was designed to answer three focal questions: (1) what happened in these investigations and how thorough was the police investigative response?; (2) to what extent did police provide explanations for their investigative response and what specific beliefs and attitudes appeared in their explanations?; and (3) to what extent did the investigative responses and explanations provided vary across cases?. To answer these questions, the process evaluation took the form of a sequential exploratory mixed methods multi-study. In this approach, the multi-study sequence begins with exploratory qualitative work to learn about the phenomenon of interest. Findings from this phase of work are then used to inform the subsequent quantitative investigation. In this program, process evaluation, directed, and conventional content analyses were used to code actual police case records for explanations provided by police for their response to sexual assault. Then, path analysis was used to examine empirically the relationships between the investigative response, the explanations provided by police for the investigative response, and specific case variables. Briefly, path analysis is a flexible approach that allows for the empirical examination of theorized models that explain how variables relate to one another by constraining and relaxing parameters within the model, assessing for improved model fit along the way. Evaluators then used analytic method to examine the explanations provided by police as predictors of the police response; the relationship between the different types of explanations provided by police and the police response; and the impact of victim and perpetrator sex, race, and age, as well as the number of perpetrators, on both the explanations provided by police and the police response.

Correct Answer:

Verified

Cousins emphasizes that engaging multiple stakeholders in a participatory manner and focusing on local, immediate needs will help stakeholders become invested in the evaluation and, therefore, they will be more likely to use the findings to promote change. Cousins defines multiple types of "use" that could result from a practical participatory evaluation project. First, process use refers to the changes within program staff (e.g., knowledge, skill development) and organizations (e.g., culture shifts) that stem from participating in an evaluation. This type of use is often referred to as the development of "evaluative thinking." Second, instrumental use is when the evaluation findings are directly used to inform a decision or contribute to solving a problem. In this type of use, there must be a clear, discernable link between the substantive results and a programmatic change (e.g., "because we found this, we decided to do that"). Finally, conceptual use is when the evaluation findings change how stakeholders think about a program or policy in a more general way (e.g., a new insight, a deeper understanding, new questions). In practical participatory evaluations, the goal is that all forms of use (process, instrumental, and conceptual) occur, but in a policy context, there is particular emphasis on promoting instrumental and conceptual use, as they are critical for policy change.
For instrumental and conceptual use to occur, policy makers need to be aware of evaluation findings and their implications. In practical participatory evaluations, stakeholders have been involved in the process of generating the work, but that is still no guarantee that the findings will be acted upon to generate change. To that end, bilateral models of engagement call for researchers to be embedded in policy work. This same idea has been percolating in policy studies since the 1990s when the concept of participatory policy analyses was introduced as a way of making public policy more relevant and responsive to needs of citizenry. It has been argued that public participation is critical to the success of a policy, and the scientific community is one sector of the public that often has relevant, empirically based information to guide policy development. Thus, the collaboration between researchers, practitioners, and policy makers must extend beyond the evaluation itself and create ways for researchers/evaluators to continue to be engaged throughout the next phases of policy development and implementation. This engagement can take many forms (e.g., participation in policy work groups, providing testimony in legislative hearings), and identifying and sharing successful strategies of such collaborations remains a pressing need in policy scholarship.