Exam 7: Evaluation Capacity Building: A Partnership Approach for Program Improvement and Social Change

Correct Answer:

Verified

An empirically validated model and corresponding evaluation capacity assessment instrument (ECAI) developed by Taylor-Ritzler, Suarez-Balcazar, and their colleagues (2013) includes important indicators of evaluation capacity such as the degree to which evaluation is conducted routinely (mainstreamed) within the organization across programs, and evaluation findings are used to make program improvements. Further, Taylor-Ritzler et al. (2013) identified specific individual and organizational indicators of evaluation capacity that, in interaction, are predictive of evaluation mainstreaming and use. The individual factors include attitudes, motivation, and knowledge and skills. Examples of the individual indicators are: (responses will vary)
Attitudes: Perceptions of evaluation (benefits, importance, drawbacks, uses) and myths
Motivation: Motivation to learn about, conduct, and support others to engage in evaluation; readiness; and importance
Knowledge and skills: Knowledge about evaluation plan, logic models, outcomes and outputs, how to collect data, analyze data, develop recommendations, and write an evaluation report
The organizational factors include leadership, learning climate, and resources. Examples of the organizational indicators are: (responses will vary)
Leadership: Leadership style; communication; leaders consider staff concerns, have realistic expectations of what staff can accomplish; management policies; commitment to evaluation; clearly articulated goals; managers supporting individuals and the organization to engage in evaluation
Learning Climate: Reflection, discussion, problem-solving, respect; opportunities for feedback and brainstorming
Resources: Accommodating the needs of people with disabilities and people from diverse ethnic/racial backgrounds; resources are allocated for evaluation related to:
Funding, time, people, space
Hardware, software, data management system
Training, consultation and technical assistance

Correct Answer:

Verified

It is important for evaluators to recognize that partnerships take time and effort and that the long-lasting relationship with community-based organizations (CBOs) facilitated the engagement in evaluation activities. When such a strong relationship is not present, evaluators may need to spend time building trust with the CBO. A relationship can be built by getting to know the agency (staff, participants, and programs), offering voluntary services, visiting the agency, attending community events sponsored by the agency, and having informal conversations about the issues the agency faces when evaluating and documenting the impact of what they do. Partnerships between evaluators and CBOs present a rich opportunity for creating evaluation-learning communities. However, evaluation-learning communities have little room for linear thinkers. Building and sustaining evaluation capacity at the organizational level is not necessarily an easy or straightforward task, given the challenges that CBOs experience such as staff turnover, budget cuts, priorities often dictated by state or funder regulations, and demographic changes, among others. These challenges often require that the evaluators and their students go with the flow, are open minded, and are ready to accommodate last-minute changes to plans, training agendas, and project expectations. Building evaluation-learning communities also requires partners assuming different roles at different times in the process. Evaluation partners truly invested in creating evaluation capacity need to be ready to wear different hats and be flexible enough to change as needed. Being able to redefine boundaries and assume different roles requires flexibility and a collaborative participatory approach to the partnership. It is important to value and recognize the set of resources that each partner brings to the evaluation capacity building. Recognizing, identifying, and celebrating resources and expertise brought by each partner aids in creating a balanced partnership.
Overall, partnerships between evaluators and CBOs can become galvanizing movements that foster evaluation capacity building learning communities and organizational transformation. Therefore, the cup will never be filled. There is no evaluation threshold to be achieved. There is always room for improving evaluation activities and building on the experience of putting into practice what is learned as the staff engages in evaluation. Mainstreaming evaluation practices results in a cycle of improving evaluation processes and evaluation outcomes in order to meet program goals and improve the lives of individuals with disabilities. Indeed, partners can build learning communities dedicated to documenting programs and utilizing findings that foster program improvement and social change. Ongoing brainstorming and reflection among partners must take place for the cycle to continue and for the learning community to thrive.

Correct Answer:

Verified

Evaluation capacity refers to a set of competencies and practices related to the ongoing documentation of programs and use of data to inform understanding and improve the outcomes of social programs. Early definitions of evaluation capacity noted both the complexity and range of the factors to consider, as well as indicators of these. For example, it has been mentioned that evaluation capacity consists of organizational, networking, programmatic, and political components that are in place to facilitate and promote program improvement. In addition to this, others have defined evaluation capacity as ". . . intentional work to continuously create and sustain overall organizational processes that make quality evaluation and its uses routine." Overall, these early definitions indicated that the desired outcome of evaluation capacity is that evaluation is an ongoing day-to-day activity in the organization. What's more, evaluation capacity is evident when organizations engage staff in ongoing documentation activities and use findings to improve their programs.
Several comprehensive models of evaluation capacity building (ECB) are available in the literature that further define evaluation capacity the most cited of these multidisciplinary models of ECB, focuses on the interaction between evaluation knowledge, skills and attitudes, and the transfer of learning through sustainable evaluation practices. A synthesis of the ECB literature found that there is consensus that both individual and organizational factors are important indicators of evaluation capacity. Despite the presence of published ECB models, only a few are empirically validated and complemented with a corresponding assessment tool used to identify and measure indicators of evaluation capacity.