Dr. Chyung designed this course and teaches it each semester (SP & FA). It is a semester-long, project-based course.
Course Description
Students learn methods of inquiry and analysis to evaluate the effectiveness of instructional or performance improvement programs. They explore models of both formative and summative evaluations and ways to pursue such research activities. Students will gain hands-on experience, actually conducting evaluation. PREREQ: OPWL536
Course Goal
The goal of OPWL 530 is to help students become competent practitioners who are capable of determining the ‘values’ of performance improvement interventions and continue to be educated consumers of evaluation findings. To do so, this course is designed to help students (1) acquire knowledge about how to conduct evaluation projects, (2) apply evaluation-specific logic and methodology to evaluation projects, (3) analyze the evaluation results using quantitative and qualitative methods, (4) report synthesized results with evaluative conclusions, and (5) recommend future action plans to the clients.
Textbooks Used
Overall Procedure to Follow
Students complete a client-based evaluation project by completing the following ten steps (Chyung, 2019):
Macro-level: "Assess feasibility, risks, and ethical concerns throughout the steps."| Micro-level | Ten Steps | During the Step | Deliverable |
|---|---|---|---|
| 1 | Identify an evaluand. | Meet with the client to understand what needs to be evaluated. | Submit a statement of work to the stakeholders, and continue with evaluation planning or decline the evaluation request. |
| 2 | Identify stakeholders of the program and their needs. | Understand stakeholders’ needs which helps identify the purpose of evaluation. | Submit a statement of work to the stakeholders, and continue with evaluation planning or decline the evaluation request. |
| 3 | Identify the purpose of evaluation based on how the evaluation findings will be used. | Steps 1, 2 and 3 may run in parallel to finalize the purpose of evaluation based on stakeholders’ needs. | Submit a statement of work to the stakeholders, and continue with evaluation planning or decline the evaluation request. |
| 4 | Develop or review a program logic model for the program. | Involve the client and other stakeholders. | Submit an evaluation proposal to the stakeholders, and continue with the implementation phase or forgo/delay it. |
| 5 | Determine dimensions and importance weighting. | Align dimensions with the stakeholders’ needs and their intention to use evaluation findings. | Submit an evaluation proposal to the stakeholders, and continue with the implementation phase or forgo/delay it. |
| 6 | Determine data collection methods. | Plan to use primary data and multiple datasets whenever possible. | Submit an evaluation proposal to the stakeholders, and continue with the implementation phase or forgo/delay it. |
| 7 | Develop data collection instruments. | Obtain approval from the stakeholders. | Submit an evaluation final report to the stakeholders and assist their use of the evaluation findings. |
| 8 | Collect data. | Maintain confidentiality of data. | Submit an evaluation final report to the stakeholders and assist their use of the evaluation findings. |
| 9 | Analyze data against rubrics. | Triangulate multiple data sources, and check with the stakeholders regarding appropriateness of rubrics. | Submit an evaluation final report to the stakeholders and assist their use of the evaluation findings. |
| 10 | Synthesize dimensional results and draw conclusions. | Organize findings based on the stakeholders’ needs and their intention to use evaluation findings. | Submit an evaluation final report to the stakeholders and assist their use of the evaluation findings. |