UC Davis California Communities Program

Sierra Health Foundation has contracted with a research team led by the California Communities Program at UC Davis to evaluate the REACH Community Action initiative. Sierra Health is providing almost $8 million to support youth development in the Greater Sacramento region with approximately $4.6 million going to seven community coalitions. The goals are to build community capacity and conditions for change and to implement strategies that increase meaningful supports and opportunities for youth ages 10-15, leading to positive youth development. Youth engagement is a key principle guiding the initiative and the local coalitions are actively working to involve youth directly in their planning processes and community change strategies.

Purpose of the evaluation
Our evaluation is grounded in systematic inquiry about REACH implementation and outcomes.
The purpose of the evaluation is:

to examine the degree to which the REACH initiative accomplished its intended goals;
to promote learning that is relevant to the needs of REACH initiative stakeholders and to the fields of community and youth development; and
to help inform future Sierra Health Foundation investment decisions.

Key research questions
During the evaluation design phase, we asked Sierra Health representatives, local coalition coordinators and technical assistance providers what questions they wanted the evaluation to address. Drawing on their responses and on the “Community Action Framework for Youth Development” (Youth Development Strategies, Inc.), we identified a set of key research questions about REACH implementation and outcomes. The focus of our external evaluation will be on five outcome questions:

Does REACH nurture viable coalitions and enhance community capacity to support youth development?
Does REACH engage youth in ways that a) enhance young people’s skills, leadership and community involvement, and b) build community capacity to support youth development?
Does REACH catalyze meaningful community change strategies related to youth development?
Does REACH increase meaningful supports and opportunities for youth in local communities?
Does REACH suggest lessons for foundation practice and the broader fields of community and youth development?

Research process
The evaluation approach will emphasize:

comparing experiences across the local settings;
developing case studies that describe individual coalition activities and outcomes (drawing on local self-evaluation data and our own fieldwork);
documenting outcomes at the individual (youth and adult), organizational and community systems levels (using both narrative and GIS mapping techniques); and
casting a wide net to include multiple ways in which local coalitions may interpret their mission and create outcomes of value (rather than judging them against pre-set criteria).

The primary means of gathering information will include:

Interviews: semi-structured, confidential interviews during the initiative with:
- site coordinators
- adult members of coalitions
- youth members of coalitions
- other youth in the community
- informed local observers not directly connected with the coalition (e.g., local Cooperative Extension advisor, elected official, head of social services department)
- Sierra Health Foundation program staff and REACH technical assistance providers
- others identified as important informants during the course of our work
Observations: of a sample of coalition meetings and activities and of initiative-wide events
Document review: written materials related to local coalition work, including
- reports/plans/logic models/self-evaluations submitted to Sierra Health
- meeting minutes
- membership/volunteer lists
- budgets/job descriptions
- flyers, publicity, media stories, etc.
- records of foundation contacts with regional institutions
Using digital storytelling to engage youth as co-researchers: We will train selected youth and their adult partners to document/reflect on their REACH and community experiences. We expect that the products of this process can be used to inform both the coalition self-evaluations and the external evaluation.
Youth survey: Working in cooperation with the site coordinators and the evaluation technical assistance team, we will design a survey that will be taken by youth participating in the seven coalitions. The survey will provide information that will inform both the coalition self-evaluations and the external evaluation. We hope to field the survey once a year during all three years of the initiative.
Review of secondary data: During our evaluation, secondary data will be used primarily to help establish the context in which local coalitions are working. Over the long-term (perhaps beyond the three years funded by the current initiative), secondary data may be helpful in tracking changing conditions for youth and their communities.

Other data-gathering activities will be developed in response to emerging information needs, local adaptations, or the evolution of the initiative.

External evaluation products will include:

periodic updates on evaluation findings;
findings from the youth survey and youth digital stories;
interim (December 2008) and final (November 2010) reports that summarize the activities and outcomes of each coalition and use a comparative analysis to draw lessons across the cases;
(where possible) coalition-specific products such as GIS maps that document coalition networks and the reach of coalition activities.

The evaluation team
The evaluation is being designed and conducted by a UC Davis-led research team with experience in the fields of community and youth development and in conducting evaluations of collaborative community change initiatives. Team members are:
David Campbell, Principal Investigator
Nancy Erbstein, Co-Principal Investigator
Leslie Cooksy
James Fabionar
Melanie Moore Kubo
Cathy Lemp

For more information, contact:
David Campbell
Department of Human and Community Development
University of California
One Shields Avenue
Davis, CA 95616
530-754-4328
dave.c.campbell@ucdavis.edu