“Are We Doing It Right?”: The Dynamic CIPP Governance Model Evaluation Tool for Program Effectiveness
Keywords:
Confirmatory factor analysis, Dynamic CIPP Governance Model, Instrument development, Local government evaluation, Scale validationAbstract
Local government units require context-sensitive evaluation tools that incorporate beneficiaries’ experiences into assessments of local health initiatives. However, the reviewed literature revealed a gap in culturally and linguistically adapted instruments for assessing beneficiaries’ perceptions across the Context, Input, Process, and Product dimensions in Philippine local-government settings. This study developed and evaluated the Dynamic CIPP Governance Model Evaluation Tool using a sequential, two-phase design. Phase 1 involved participatory cultural and linguistic adaptation, pretesting, internal-consistency analysis, and exploratory factor analysis with 125 beneficiaries, parents or guardians of child beneficiaries, and local program personnel. Findings informed the refinement of selected items. In Phase 2, 701 respondents from two local health initiatives completed the revised 28-item instrument, comprising parents or guardians of child beneficiaries in one initiative and direct beneficiaries in the other. Its hypothesized correlated four-factor structure was evaluated through confirmatory factor analysis. Cronbach’s alpha and McDonald’s omega ranged from 0.88 to 0.90 across the four dimensions. The model demonstrated good fit, χ²(344) = 360.29, p = .262, comparative fit index = 0.998, Tucker-Lewis index = 0.998, root mean square error of approximation = 0.01, and standardized root mean square residual = 0.02. Standardized loadings ranged from 0.66 to 0.79. Average variance extracted ranged from 0.50 to 0.56, composite reliability from 0.88 to 0.90, and heterotrait-monotrait ratios from .43 to .52. The results supported the internal consistency, factorial validity, convergent validity, composite reliability, and discriminant validity of the four dimensions. The tool therefore provides local government units with a structured, beneficiary-oriented means of complementing administrative and outcome evidence when evaluating local health initiatives.
Downloads
References
Acenas, A. (2023). Evaluation of undergraduate research course using Stufflebeam’s context, input, process, product model. Journal of Interdisciplinary Perspectives, 1(1), 60–69. https://doi.org/10.69569/jip.2023.0003
Aizpurua, E. (2020). Pretesting methods in cross-cultural research. In M. Sha & T. Gabel (Eds.), The Essential Role of Language in Survey Research (pp. 129–150). RTI International. https://doi.org/10.3768/rtipress.bk.0023.2004
Ajzen, I. (1991). The theory of planned behavior. Organizational Behavior and Human Decision Processes, 50(2), 179–211. https://doi.org/10.1016/0749-5978(91)90020-T
Ajzen, I., & Fishbein, M. (2014). The influence of attitudes on behavior. In D. Albarracin, B. T. Johnson, & M. P. Zanna (Eds.), The Handbook of Attitudes (pp. 173–221). Psychology Press. https://doi.org/10.4324/9781410612823 (Original work published 2005, Lawrence Erlbaum Associates Publishers)
Altarriba, J. (1992). The representation of translation equivalents in bilingual memory. Advances in Psychology, 83(C), 157–174. https://doi.org/10.1016/S0166-4115(08)61493-4
Amoto, J., Sodusta, D.J., Fernandez, Jr., P., Firmase, J., & Nanta, E.M. (2024). Evaluating the University of the Philippines Visayas Action Research Project based on the context, input, process, and product model. Davao Research Journal, 15(4), 120–130. https://doi.org/10.59120/drj.v15i4.280
Balgos, B. (2023). NEPF summative evaluation of the National Evaluation Policy Framework—Final report (pp. 1–72) [Evaluation report]. Department of Economy, Planning, and Development. https://nep.depdev.gov.ph/evaluations/105
Betsch, C., Böhm, R., & Chapman, G. (2015). Using behavioral insights to increase vaccination policy effectiveness. Policy Insights from the Behavioral and Brain Sciences, 2(1), 61–73. https://doi.org/10.1177/2372732215600716
Catubig, R.M.C. (2026). School-based nutrition programs in Philippine basic education: A context input process and product (CIPP)-based evaluation model. International Journal of the Humanities and Social Sciences, 5(2), 5860–5887. https://doi.org/10.69651/2022PIJHSS
Çelebi, I., Gökkaya, E., Çopur, A., & Balci, E. (2025). Quiet quitting reason and behavior scale design and development process. Work, 80(1), 164–172. https://doi.org/10.3233/WOR-230699
Corpus, V. (2022). The Philippines’ National Evaluation Policy Framework, evaluation practice, challenges and prospects [Policy framework presentation document]. National Economic and Development Authority. https://tinyurl.com/yfctxrp6
Davis, F. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13(3), 319–338. https://doi.org/10.2307/249008
dela Luna, K.L., Sy, A.D., Sabandal, R.L.C., Jacinto, C.B., Malimban, R., Sales, B.E., Marollano, R., & Marin, A. (2026). A qualitative program evaluation study on the perceived impact of health and nutrition programs among beneficiaries of a civil society organization in the Philippines. Acta Medica Philippina. https://doi.org/10.47895/amp.vi0.12745
Ding, J., Wang, A., & Zhang, Q. (2023). Mining the vaccination willingness of China using social media data. International Journal of Medical Informatics, 170, 104941. https://doi.org/10.1016/j.ijmedinf.2022.104941
Drucker, P.F. (2006). The effective executive: The definitive guide to getting the right things done. Harper Business.
Ertay, Z., & Gilanlioglu, I. (2024). The development and validation of an intercultural communicative competence scale for students. SAGE Open, 14(2), 1–11. https://doi.org/10.1177/21582440241252495
Fornell, C., & Larcker, D. (1981). Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research, 18(1), 39–50. https://doi.org/10.1177/002224378101800104
Hair, J.F. Jr., Hult, G.T.M., Ringle, C.M., & Sarstedt, M. (2022). A primer on partial least squares structural equation modeling (PLS-SEM) (Third Edition). SAGE Publications, Inc.
Henseler, J., Ringle, C., & Sarstedt, M. (2015). A new criterion for assessing discriminant validity in variance-based structural equation modeling. Journal of the Academy of Marketing Science, 43(1), 115–135. https://doi.org/10.1007/s11747-014-0403-8
Khoury, B., & Vergara, R. (2026). Compassion questionnaires revised: Scales development and validation. Assessment, 33(4), 653–687. https://doi.org/10.1177/10731911251337185
Kline, R.B. (2016). Principles and practice of structural equation modeling (4th Ed.). The Guilford Press.
Luib, S., Loquias, M., Capule, F., Garcia, K.E., & Sendito, M.J. (2022). Evaluation of the patient medication counseling services in the Philippine General Hospital using the CIPP model. Acta Medica Philippina, 56(6), 143–150. https://doi.org/10.47895/amp.v56i6.4522
McCormick, J.S. (1981). Effectiveness and efficiency. Journal of the Royal College of General Practitioners, 31(226), 299–302. https://pmc.ncbi.nlm.nih.gov/articles/PMC1971043/
Metehan, C.T., Sözer, E.G., Civelek, M.E., Ertemel, A.V., & Mert, I.S. (2025). Buyer platform experience (PLX) in online marketplaces: Scale development, validation, and the salience of variety, competitive pricing, service excellence, and promotion. SAGE Open, 15(4), 1–20. https://doi.org/10.1177/21582440251394446
Municipality of Tudela. (2024). “Are we doing it right?”: Decoding local health initiatives through the lens of program effectiveness evaluation [Approved research activity proposal]. Office of the Municipal Administrator, Municipality of Tudela.
National Economic and Development Authority [NEDA]. (2020). Guidelines on evaluation in the national government (pp. 1–94) [Guidelines document]. NEDA.
Nunnally, J.C. (1978). Psychometric theory (2nd Edition). McGraw-Hill, Inc.
Office of the Mayor of Tudela. (2024). Executive Order No. 19-B series of 2024: An order creating and mandating an activity team to conduct the scientific study “Are we doing it right?”: Decoding local health initiatives through the lens of program effectiveness (Executive Order No. 19‑B Series of 2024). Municipality of Tudela.
Peytcheva, E. (2020). The effect of language of survey administration on the response formation process. In M. Sha & T. Gabel (Eds.), The Essential Role of Language in Survey Research (pp. 3–21). RTI International. https://doi.org/10.3768/rtipress.bk.0023.2004
Phang, G., & Ilham, Z. (2023). Theory of planned behavior to understand pro-environmental behavior among Universiti Malaya students. AIMS Environmental Science, 10(5), 691–708. https://doi.org/10.3934/environsci.2023038
Prasetyo, Y.T., Castillo, A.M., Salonga, L.J., Sia, J.A., & Seneta, J.A. (2020). Factors affecting perceived effectiveness of COVID-19 prevention measures among Filipinos during enhanced community quarantine in Luzon, Philippines: Integrating protection motivation theory and extended theory of planned behavior. International Journal of Infectious Diseases, 99, 312–323. https://doi.org/10.1016/j.ijid.2020.07.074
Repke, L., & Felderer, B. (2025). The relationship between measurement error, representation bias, language, and country: A comparative analysis using the European Social Survey (Rounds 5 to 7). Survey Methods: Insights from the Field, Special Issue ‘Advancing Comparative Research: Exploring Errors and Quality Indicators in Social Research.’ https://doi.org/10.13094/SMIF-2025-00013
Rivera, A.L.S., & Altarriba, J. (2002). The role of language in therapy with the Spanish-English bilingual client. Professional Psychology: Research and Practice, 33(1), 30–38. https://doi.org/10.1037/0735-7028.33.1.30
Rodrigues, F., Ziade, N., Jatuworapruk, K., Caballero-Uribe, C.V., Khursheed, T., & Gupta, L. (2023). The impact of social media on vaccination: A narrative review. Journal of Korean Medical Science, 38(40), e326. https://doi.org/10.3346/jkms.2023.38.e326
Rossi, P.H., Lipsey, M.W., & Freeman, H.E. (2004). Evaluation: A systematic approach (7th Ed.). SAGE Publications, Inc.
Sankaran, S., & Saad, N. (2022). Evaluating the Bachelor of Education program based on the context, input, process, and product model. Frontiers in Education, 7, 924374. https://doi.org/10.3389/feduc.2022.924374
Seddig, D., Maskileyson, D., Davidov, E., Ajzen, I., & Schmidt, P. (2022). Correlates of COVID-19 vaccination intentions: Attitudes, institutional trust, fear, conspiracy beliefs, and vaccine skepticism. Social Science & Medicine, 302, 114981. https://doi.org/10.1016/j.socscimed.2022.114981
Social Weather Stations [SWS]. (2025). Fourth Quarter 2024 Social Weather Survey: Self-rated poverty (SRP) at 63% in December 2024, up from 59% in September; borderline at 11%, not poor at 26% [Social weather report]. Social Weather Station. https://sws.org.ph/social-weather-report-fourth-quarter-2024-social-weather-survey-self-rated-poverty
Stufflebeam, D. (2003). The CIPP model for evaluation. In T. Kellaghan, D. L. Stufflebeam, & L. A. Wingate (Eds.), International Handbook of Educational Evaluation Part 1 (Vol. 9, pp. 31–62). Kluwer Academic Publishers. https://doi.org/10.1007/978-94-010-0309-4_4
Sundari, S. (2019). Implementation CIPP model for exploring evaluation of analysis performance policy to Ministry Defense Indonesia (MDI). International Journal of Scientific & Technology Research, 8(10), 1417–1423. https://tinyurl.com/3xe866yu
Tabachnick, B.G., & Fidell, L.S. (2014). Using multivariate statistics (Sixth Edition, Pearson New International Edition). Pearson.
United Nations General Assembly. (2015). Transforming our world: The 2030 agenda for sustainable development (General Assembly Resolution A/RES/70/1). United Nations. https://digitallibrary.un.org/record/3923923/
U.S. Government Accountability Office [GAO]. (2021). Program evaluation: Key terms and concepts. U.S. Government Printing Office. https://www.gao.gov/assets/gao-21-404sp.pdf
Wahjusaputri, S., & Irawan, B. (2019). Bureaucratic reform policy strategy using the CIPP model for the improvement of public services in Kemenko-Polhukam. Jurnal Ekonomi Dan Bisnis, 3(1), 1–16. https://journal.uhamka.ac.id/index.php/agregat/article/view/3013
Wandani, F.T., Widodo, A., Rubiyanti, Nurafni, Rubiyanti, N., & Silvianita, A. (2024). Analysis of the influence of environmental concern, theory of planned behavior, and theory of reasoned action on purchase intention: A conceptual paper. International Journal of Scientific Multidisciplinary Research, 2(5), 499–508. https://doi.org/10.55927/ijsmr.v2i5.9138
Zhang, G., Zeller, N., Griffith, R., Metcalf, D., Williams, J., Shea, C., & Misulis, K. (2011). Using the context, input, process, and product evaluation model (CIPP) as a comprehensive framework to guide the planning, implementation, and assessment of service-learning programs. Journal of Higher Education Outreach and Engagement, 15(4), 57–83. https://openjournals.libs.uga.edu/jheoe/article/view/901
Zhang, Y., & Aryadoust, V. (2022). A systematic review of the validity of questionnaires in second language research. Education Sciences, 12(723), 1–32. https://doi.org/10.3390/educsci12100723
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
JIP is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License. Under the Open Access Policy, appropriate attribution can be provided by simply citing the original article.
