A program’s infrastructure is often cited as critical to public health success. The Component Model of Infrastructure (CMI) identifies evaluation as essential under the core component of engaged data. An evaluation plan is a written document that describes how to monitor and evaluate a program, as well as how to use evaluation results for program improvement and decision making. The evaluation plan clarifies how to describe what the program did, how it worked, and why outcomes matter. We use the Centers for Disease Control and Prevention’s (CDC) “Framework for Program Evaluation in Public Health” as a guide for developing an evaluation plan. Just as using a roadmap facilitates progress on a long journey, a well-written evaluation plan can clarify the direction your evaluation takes and facilitate achievement of the evaluation’s objectives.
A program’s infrastructure is often cited as a critical component of public health success. 1,2 The Component Model of Infrastructure (CMI) identifies evaluation as a critical component of program infrastructure under the core component of engaged data. 3 A written evaluation plan that is thoughtful, transparent, and collaboratively developed is the preferred method for effective evaluation planning. 4
The Office on Smoking and Health at the Centers for Disease Control and Prevention (OSH/CDC) has a long history of supporting evaluation and evaluation capacity building as a central component of state tobacco control program infrastructure. 5–7 For example, a written evaluation plan was included as a requirement for states who applied for the funding opportunity announcement for the Communities Putting Prevention to Work cooperative agreements. The CDC’s expectation was that funded programs would develop functional evaluation plans to guide the implementation of useful evaluations of their Communities Putting Prevention to Work activities. In order to promote evaluation capacity building, an evaluation plan development workbook was created by OSH and the Division of Nutrition, Physical Activity, and Obesity. 4 The workbook is a how-to guide intended to assist public health program managers, administrators, health educators, and evaluators in developing a joint understanding of what constitutes an evaluation plan, why it is important, and how to develop an effective evaluation plan. The workbook guides the user through the plan development process using the 6 steps of the CDC’s “Framework for Program Evaluation in Public Health” 8 and provides tools, worksheets, and a resource list. Although the workbook was collaboratively created by OSH and the Division of Nutrition, Physical Activity, and Obesity to provide guidance to tobacco and obesity prevention programs, it can be used by a broad public health audience to guide their evaluation plan development process.
Area 4 of the 7 Certified Health Education Specialists’ responsibilities 9 is “conduct evaluation and research related to health education.” (p4) More specifically, Competency 4.1, “develop an evaluation/research plan,” provides a list of 14 subcompetencies or steps in developing an evaluation plan. 9 (p4) This easy-to-read, abridged version of the workbook addresses these competencies in a concrete manner that can increase health education specialists’ professional capacities. This article summarizes the major steps in the evaluation plan development process and provides relevant examples for health educators. It may be used alone to develop an evaluation plan. We refer the reader to the workbook for more in-depth instruction, tools, worksheets, and resource suggestions. 4
An evaluation plan is a written document that describes how you will monitor and evaluate your program, as well as how you intend to use evaluation results for program improvement and decision making. The evaluation plan clarifies how you will describe the “what,” the “how,” and the “why it matters” for your program.
The “what” describes your program and how its activities are linked to its intended effects. It serves to clarify the program’s purpose and anticipated outcomes.
The “how” addresses the process for implementing a program and provides information about whether the program is operating with fidelity to the program’s design.
The “why it matters” provides the rationale for your program and its intended impact on public health. This is also sometimes referred to as the “so what?” question. Being able to demonstrate that your program has made a difference is critical to program sustainability.
An evaluation plan is similar to a roadmap. It clarifies the steps needed to assess the processes and outcomes of a program. An effective evaluation plan is more than a list of indicators in your program’s work plan. It is a dynamic tool that should be updated on an ongoing basis to reflect program changes and priorities over time.
Just as using a roadmap facilitates progress on a long journey, an evaluation plan can clarify the direction of your evaluation based on the program’s priorities and resources and the time and skills needed to accomplish the evaluation. The process of developing a written evaluation plan in cooperation with an evaluation stakeholder workgroup (ESW) will foster collaboration; give a sense of shared purpose to the stakeholders; create transparency through the implementation process; and ensure that stakeholders have a common vision and understanding of the purpose, use, and users of the evaluation results. The use of evaluation results must be planned, directed, and intentional and should be included as part of the evaluation plan. 10
There are numerous ways in which you can frame your evaluation plan. We use the CDC’s “Framework for Program Evaluation in Public Health” as a guide for the planning process and outlining considerations for what to include in the written evaluation plan. 8 The CDC framework is a guide on how to effectively evaluate public health programs and on using your evaluation’s findings for program improvement and decision making ( Figure 1 ). There are countless ways to organize your evaluation plan. Using the framework to organize your plan will facilitate including concrete elements that promote transparent and thoughtful implementation of the evaluation. The workbook and this abridged version of the workbook provide needed resources on how to use the framework for implementing an evaluation to develop the plan itself. Though the framework is described in terms of steps, the actions are not always linear and are often completed in a cyclical nature. The development of an evaluation plan is an ongoing process; you may need to revisit a step during the process and complete several steps concurrently.
The CDC framework for program evaluation (color figure available online).
A primary feature of an evaluation plan is the identification and acknowledgement of the roles and responsibilities of an ESW. The ESW includes members who have a stake or vested interest in the evaluation findings and those who are the intended users of the evaluation. 10,11 The ESW may also include others who have a direct or indirect interest in program implementation. Engaging stakeholders in the ESW enhances intended users’ understanding and acceptance of the utility of evaluation information. Stakeholders are much more likely to buy into and support the evaluation if they are involved in the evaluation process from the beginning.
For the ESW to be truly integrated in the development of the evaluation plan, ideally, it will be identified in the evaluation plan. The form this takes may vary based on program needs. If it is important politically, a program might want to specifically name each member of the workgroup, their affiliation, and specific role(s) in the workgroup. Being transparent about the role and purpose of the ESW can facilitate buy-in for the evaluation plan. In addition, you may want to include the preferred method of communication and the timing of that communication for each stakeholder or group. A stakeholder chart or table can be a useful tool to include in your evaluation plan.
The next step in the evaluation plan is to describe the program. A program description clarifies the program’s purpose, stage of development, activities, capacity to improve health, and implementation context. A shared understanding of the program by health educators, program staff, evaluators, and the ESW and what the evaluation can and cannot deliver is essential to implementation of evaluation activities and use of evaluation results. A narrative description in the written plan is helpful to ensure a full and complete shared understanding of the program and a ready reference for stakeholders. A logic model may be used to succinctly synthesize the main elements of a program. The program description is essential for focusing the evaluation design and selecting the appropriate methods. Too often groups jump to evaluation methods before understanding what the program is designed to achieve or what the evaluation should deliver. The description will be based on your program’s objectives and context but most descriptions include at a minimum:
A statement of need to identify the health issue addressed Inputs or program resources needed to implement program activities Program activities linked to program outcomes through theory or best practice program logic Stage of development of the program to reflect program maturity Environmental context within which the program is implementedIn terms of describing the stage of development of the program, the developmental stages that programs typically move through are planning, implementation, and maintenance. For policy or environmental initiatives, which programs and health educators often evaluate, the stages might look somewhat like this:
