Ensuring Validity and Reliability in Algebra Midterm Assessment: A Comprehensive Approach to Test Development and Analysis
DOI:
https://doi.org/10.69569/jip.2024.0497Keywords:
College Algebra, Advanced Algebra, Item analysis, Mathematics education, Test developmentAbstract
First-year mathematics education students often face challenges with College and Advanced Algebra assessments. This study aimed to develop and validate a 100-item multiple-choice midterm test for College and Advanced Algebra, ensuring the test's validity and reliability. The test was designed following a structured process that included creating a Table of Specifications (TOS) based on the approved syllabus. To ensure content accuracy and relevance, the test was reviewed by three subject matter experts and evaluated for clarity by 15 students. Pilot testing was conducted with 82 fourth-year Bachelor of Secondary Education Mathematics (BSEd Mathematics) students. The pilot test results underwent detailed item analysis, focusing on metrics such as the difficulty index, discrimination index, and overall reliability using the Kuder-Richardson Formula 20 (KR-20). Of the 100 items, 22 were retained, 48 were revised, and 30 were discarded. The test achieved a reliability coefficient of 0.876, indicating strong internal consistency. The findings suggest that the validated test questionnaire is a dependable tool for accurately assessing students’ knowledge in College and Advanced Algebra, providing valuable feedback for educators and students.
Downloads
References
Aguhayon, H., Tingson, R., & Pentang, J. (2023). Addressing students learning gaps in mathematics through differentiated instruction. International Journal of Educational Management and Development Studies, 4(1), 69–87. https://doi.org/10.53378/352967
Anderson, L., & Krathwohl, D.A. (2001). Taxonomy for learning, teaching and assessing: A revision of Bloom's Taxonomy of Educational Objectives. New York: Longman.
Andreev, I. (2024). Bloom’s Taxonomy. Retrieved from https://tinyurl.com/42pmmk9b
Biddix, J.P. (2018). Research methods and applications for student affairs. John Wiley & Sons.
Bilyakovska, O. (2022). Test as an effective means of assessing the quality of students’ knowledge. Academic Notes Series Pedagogical Science, 1(204), 16–20. https://doi.org/10.36550/2415-7988-2022-1-204-16-20
Bobbitt, Z. (2022). Kuder-Richardson Formula 20 (Definition & example). Retrieved from https://www.statology.-org/kuder-richardson-20/
Cañeda, M.E., Amar, R.P., & Lucin, E.L. (2024a). Development of test questionnaire on selected topics in calculus 1 (final term). International Journal of Research and Scientific Innovation, 9(8), 244-255. https://doi.org/10.51244/IJRSI.2024.1108020
Cañeda, M.E., Logroño, J.J.F., & Culibra, C.D. (2024b). Test questionnaire development on selected topics in calculus 1. Ignatian International Journal for Multidisciplinary Research, 2(8), 1363–1376. https://doi.org/-10.5281/zenodo.13371155
Chigonga, B. (2020). Formative Assessment in Mathematics Education in the Twenty-First Century. IntechOpen.
DeVellis, R.F. (2003). Scale development: theory and applications, applied social research methods. Sage Publications.
Dwyer, C. P., Hogan, M. J., & Stewart, I. (2014). An integrated critical thinking framework for the 21st century. Thinking Skills and Creativity, 12, 43–52. https://doi.org/10.1016/j.tsc.2013.12.004
Ferrando, P.J., Lorenzo-Seva, U., & Bargalló-Escrivà, M. T. (2023). Gulliksen's pool: A quick tool for preliminary detection of problematic items in item factor analysis. PloS one, 18(8), e0290611. https://doi.org/10.1371/journal.pone.0290611
Gilbert, G.E., & Prion, S. (2016). Making sense of methods and measurement: Lawshe’s content validity index. Clinical Simulation in Nursing, 12(12), 530–531. https://doi.org/10.1016/-j.ecns.2016.08.002
Irwing, P., & Hughes, D.J. (2018). Test development. In P. Irwing, T. Booth, & D. J. Hughes (Eds.), The Wiley handbook of psychometric testing: A multidisciplinary reference on survey, scale and test development (pp. 3–47). Wiley Blackwell.
Jain, S., Dubey, S., & Jain, S. (2016). Designing and validation of questionnaire. International Dental & Medical Journal of Advanced Research, 2(1), 1–3. https://doi.org/10.15713/ins.idmjar.39
Jhangiani, R.S., Chiang, I.A., Cuttler, C., & Leighton, D.C. (2019). Research methods in psychology – 2nd Canadian edition. KPU.
Kara, F., & Celikler, D. (2015). Development of achievement test: Validity and reliability study for achievement test on matter changing. Journal of Education and Practice, 6(24), 21–26. https://eric.ed.gov/?id=EJ1078816
Kılıç, A. & Uysal, I. (2022). To what extent are item discrimination values realistic? A new index for two-dimensional structures. International Journal of Assessment Tools in Education, 9, 728-740. https://doi.org/10.21449/ijate.1098757
Kilic, S. (2016). Cronbach’s alpha reliability coefficient. Journal of Mood Disorders, 6(1), 47. https://doi.org/10.5455/jmood.20160307122823
Kline, P. (2000). Handbook of psychological testing, second edition. Routledge.
Kunwar, R. (2018). Development and standardization process of mathematics achievement test for the students of grade x. International Journal of Current Research, 10(11), 75451–75455. https://doi.org/10.24941/ijcr.33168.11.2018
Lazarus, S.S., Johnstone, C.J., Liu, K.K., Thurlow, M.L., Hinkle, A.R., & Burden, K. (2022). An updated state guide to universally designed assessments (NCEO Report 431). Retrieved from https://tinyurl.com/y7tf3ty9
Longe, I.O., & Maharaj, A. (2023). Investigating students’ understanding of complex number and its relation to algebraic group using and APOS theory. Journal of Medives : Journal of Mathematics Education IKIP Veteran Semarang, 7(1), 117. https://doi.org/10.31331/ medivesveteran.v7i1.2332
Mamolo, L.A. (2021). Development of an achievement test to measure students' competency in general mathematics. Anatolian Journal of Education, 6(1), 79-90. https://doi.org/10.29333/aje.2021.616a
Metzgar, M. (2023). Revised Bloom’s taxonomy in a principles of Economics textbook. Acta Educationis Generalis, 13(3), 15–28. https://doi.org/10.2478/atd-2023-0019.
Ocampo, R., & Usita, N. P. (2015). Development of Lubeg (Syzygiumlineatum (Roxb.) Merr.& Perry) processed products. Asia Pacific Journal of Multidisciplinary Research, 3(4), 118-123. https://tinyurl.com/yxb9ccxn
Oducado, R. M. (2020). Survey instrument validation rating scale. Retrieved from https://doi.org/10.2139/ssrn.3789575
Padua, R.N., & Santos, R.G. (1997). Educational evaluation and measurement: Theory, practice, and application. KATHA Publishing: QC.
Patel, N., & Desai, S. (2020). Abc of face validity for questionnaire. International Journal of Pharmaceutical Sciences Review and Research, 65(1), 164–168. https://doi.org/10.47583/ijpsrr.2020.v65i01.025
Quaigrain, K., Arhin, A. K., & King Fai Hui, S. (2017). Using reliability and item analysis to evaluate a teacher-developed test in educational measurement and evaluation. Cogent Education, 4(1). https://doi.org/10.1080/2331186X.2017.1301013
Rezigalla, A. A., Ibrahim, E. K., & ElHussein, A. B. (2019). Item Analysis: The impact of distractor efficiency on the discrimination power of multiple choice items. Retrieved from https://doi.org/10.21203/rs.2.15899/v1
Stephens, A., Blanton, M., Knuth, E., Isler, I., & Gardiner, A. M. (2015). Just say yes to early algebra! Teaching Children Mathematics, 22(2), 92–101. https://doi.org/10.5951/teacchilmath.22.2.0092
Sullivan G. M. (2011). A primer on the validity of assessment instruments. Journal of graduate medical education, 3(2), 119–120. https://doi.org/10.4300/JGME-D-11-00075.1
Syahfitri, J., Firman, H., Redjeki, S., & Srivati, S. (2019). Development and validation of critical thinking disposition test in Biology. International Journal of Instruction, 12(4), 381-392. https://doi.org/10.29333-/iji.2019.12425a
Taherdoost, H. (2016). Validity and reliability of the research instrument; how to test the validation of a questionnaire/survey in a research. International Journal of Academic Research in Management, 5(3), 28-36. http://dx.doi.org/10.2139/ssrn.3205040
Tejeda, K., & Gallardo, G. (2017). Performance assessment on high school advanced algebra. International Electronic Journal of Mathematics Education, 12(3), 777-798. https://doi.org/10.29333/iejme/648
Wilson, L.O. (2016). Anderson and Krathwohl: Bloom’s taxonomy revised. Retrieved from https://tinyurl.com/4s9vhnee
Yaddanapudi, S., & Yaddanapudi, L.N. (2019). How to design questionnaires. Indian Journal of Anaesthesia, 63(5), 335–337. https://doi.org/10.4103/ija.IJA_274_19
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Journal of Interdisciplinary Perspectives
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
