Characteristic of critical thinking test instrument and analysis students’ ability using Rasch model

Siti Saffanah Apriliani; Duden Saepuzaman; Irma Rahma Suwarma

doi:10.21067/mpej.v10i1.13455

Authors

Siti Saffanah Apriliani Universitas Pendidikan Indonesia, Indonesia
Duden Saepuzaman Universitas Pendidikan Indonesia, Indonesia
Irma Rahma Suwarma Universitas Pendidikan Indonesia, Indonesia

DOI:

https://doi.org/10.21067/mpej.v10i1.13455

Keywords:

Characteristic Instrument, Critical thinking, Rasch model, Student

Abstract

This study aims to analyze the characteristics of the instrument and students' critical thinking skills on the topic of work and energy using Rasch Model. The analysis conducted includes instrument quality, instrument characteristics as indicated by person-item fit to Rasch Model, and the distribution of students' critical thinking skills. The results of the critical thinking skills test of 82 students were analyzed using Winsteps software version 3.73. The results show that the instrument used is reliable and valid, meeting the criteria for good values for both persons and items. The instrument quality analysis shows that items and persons fit Rasch Model. The Wright Map shows the pattern of students' ability distribution and the difficulty level of items, which align with the analysis of students' abilities categorized as very high, high, low, and very low, and item difficulty levels categorized as very difficult, difficult, easy, and very easy. Item Q9 is the most difficult item, and Q6 is the easiest item. The pattern of students' responses to items is shown in the Scalogram, where there are persons who provide anomalous response patterns. Overall, the instrument used has good quality for measuring students' critical thinking skills and emphasizes the importance of having a range of question difficulties.

Downloads

Download data is not yet available.

References

Abdaziz, A., Jusoh, M. S., Amlus, H., & Salleh, T. S. (2014). Construct validity: A Rasch measurement model approach. Journal of Applied Sciences Research, 9(12), 7–12.

Adams, D., Sumintono, B., Mohamed, A., & Noor, N. S. M. (2018). E-learning readiness among students of diverse backgrounds in a leading Malaysian higher education institution. Malaysian Journal of Learning and Instruction, 15(2), 227–256. https://doi.org/10.32890/mjli2018.15.2.9

Aji, S. D., Ayu, H. D., Rosita, M., & Prawira, C. A. (2024). Hybrid discovery learning: Solutions for optimizing students’ critical thinking and ICT skills on temperature fluctuations material. Momentum: Physics Education Journal, 8(1), 65–74. https://doi.org/10.21067/mpej.v8i1.8581

Boone, W. J., Staver, J. R., & Yale, M. S. (2014). Rasch analysis in the human sciences. Springer. https://doi.org/10.1007/978-94-007-6857-4

Daeng Naba, S., Istiyono, E., Kurniawan, A., & Adrianto, N. (2024). Development and validation of physics multiple-choice tests on the nature of physics using Rasch modelling analysis. Jurnal Pendidikan MIPA, 25(2), 671–685. https://doi.org/10.23960/jpmipa.v25i2.pp671-685

Ennis, R. H. (2011). The nature of critical thinking: An outline of critical thinking dispositions and abilities. University of Illinois.

Facione, P. A., & Gittens, C. A. (2015). Think critically (3rd ed.). Pearson.

Fathonah, S., Ahmadi, F., Lestari, W., Setiawan, D., & Yulianto, A. (2025). Pengembangan pembelajaran IPA terpadu berbasis keterampilan abad ke-21. Jurnal IPA Terpadu, 9(1), 37–46.

Handayani, Y., Rahmawati, R., & Widiasih, W. (2023). Using Rasch model to analyze reliability and validity of concept mastery test on electricity and magnetism topic. JIPF (Jurnal Ilmu Pendidikan Fisika), 8(2), 226–239. https://doi.org/10.26737/jipf.v8i2.3877

Herlinawati, H., Marwa, M., Ismail, N., Junaidi, J., Liza, L. O., Situmorang, D. D. B., & Situmorang, B. (2024). The integration of 21st century skills in the curriculum of education. Heliyon, 10(15), Article e35148. https://doi.org/10.1016/j.heliyon.2024.e35148

Ismail, I., Riandi, R., Kaniawati, I., Sopandi, W., & Supriyadi, S. (2024). Gender roles in understanding and implementing green energy technology in Indonesian schools: Rasch analysis. Journal of Innovation and Educational Research, 4(3), 298–314.

Jabrayilov, R., Emons, W. H. M., & Sijtsma, K. (2016). Comparison of classical test theory and item response theory in individual change assessment. Applied Psychological Measurement, 40(8), 559–572. https://doi.org/10.1177/0146621616664046

Kleppang, A. L., Steigen, A. M., & Finbråten, H. S. (2020). Using Rasch measurement theory to assess the psychometric properties of a depressive symptoms scale in Norwegian adolescents. Health and Quality of Life Outcomes, 18(1), 1–8.

Linacre, J. M. (2025). Winsteps® Rasch measurement computer program user’s guide (Version 5.10.4). Winsteps.com.

Manggaberani, A. A., & Darlis, A. M. (2024). The effectiveness of Google Forms in assessing and evaluating online learning outcomes: Meta-analysis study. International Journal of Evaluation and Research in Education, 13(10), 4561–4570.

Maulana, S., Rusilowati, A., Nugroho, S. E., & Susilaningsih, E. (2023). Implementasi Rasch model dalam pengembangan instrumen tes diagnostik. Prosiding Seminar Nasional Pascasarjana (PROSNAMPAS), 6(1), 748–757.

Nurhayati, N., Suhandi, A., Muslim, M., Kaniawati, I., Wahyudi, W., Misbah, M., & Darman, D. R. (2025). Assessing the knowledge and skills of prospective physics teachers in designing 4C skills-oriented learning: Rasch analysis. Journal of Physics Education Research, 5(1), 718–741.

Olusegun, S. (2015). Constructivism learning theory: A paradigm for teaching and learning. Journal of Research & Method in Education, 5(6), 66–70. https://doi.org/10.9790/7388-05616670

Ridwan, I. M., Kaniawati, I., Suhandi, A., Ramalis, T. R., Mahmudah, M., & Vilmala, B. K. (2023). Rasch modeling: Developing a critical thinking ability test for students on climate change material. Momentum: Physics Education Journal, 7(2), 247–258.

Rusch, T., Lowry, P. B., Mair, P., & Treiblmaier, H. (2016). Breaking free from the limitations of classical test theory: Developing and measuring information systems scales using item response theory. Communications of the Association for Information Systems, 38(1), 74–130.

Salah, S. E. (2010). Critical thinking: Reflection and perspective (Part II). Inquiry: Critical Thinking Across the Disciplines, 26(2), 5–17.

Sumintono, B., & Widhiarso, W. (2015). Aplikasi pemodelan Rasch pada assessment pendidikan. Trim Komunikata Publishing House.

Suprapto, N., Rizki, I. A., & Cheng, T.-H. (2024). Profile of students’ physics critical thinking skills and prospect analysis of project-oriented problem-based learning model. Journal of Turkish Science Education, 21(1), 134–149.

Suyono, S., Harsono, P., Sutantyo, E., & Putranto, W. A. (2025). Critical thinking and 21st-century skills: Evaluating the effectiveness of problem-based learning models in multicultural classrooms. International Journal of Educational Research Excellence, 4(2), 511–521. https://doi.org/10.55299/ijere.v4i2.1478

Wardani, I. S., & Fiorintina, E. (2023). Building critical thinking skills of 21st century students through problem based learning model. Jurnal Pendidikan Indonesia, 12(3), 461–470. https://doi.org/10.23887/jpiundiksha.v12i3.58789

Xiao, Y., Han, J., Koenig, K., Xiong, J., & Bao, L. (2018). Multilevel Rasch modeling of two-tier multiple-choice test: A case study using Lawson’s classroom test of scientific reasoning. Physical Review Physics Education Research, 14(2), Article 020104. https://doi.org/10.1103/PhysRevPhysEducRes.14.020104

Yenni, R. P., Siahaan, P., & Suwarma, I. R. (2022). ICARE-U learning model to improve critical thinking skills of high school students in Indonesia. Jurnal Pendidikan MIPA, 12(1), 1–7.

Yohanna, E., Siahaan, S., Muhammad, I., & Dasari, D. (2023). Trend of critical thinking skill researches in mathematics education in Scopus database across Indonesia: From research design to data analysis. Journal of Mathematics Education and Learning, 6(2), 151–161.

Yusliani, E., & Desnita, D. (2022). Validity and reliability of critical thinking instruments to measure the effectiveness of context-based physics e-module on wave materials. Jurnal Penelitian Pendidikan IPA, 8(1), 57–64.