RASCH model analysis: validity and reliability of concept understanding test instrument on temperature and heat

Kafa Pramitha Anggraini Indhira Artanti; Sherly Eka Putri Damayanti; Shinta Nuriyah Mahbubiyah Royani; Bintan Nuril Irvaniyah; Natasya Mustika Elvina Rossa; Anggita Meilina Putri; Khoirun Nisa'

doi:10.21067/mpej.v10i1.13156

Authors

Kafa Pramitha Anggraini Indhira Artanti Universitas Negeri Malang, Indonesia https://orcid.org/0000-0001-6100-5946
Sherly Eka Putri Damayanti Universitas Negeri Malang, Indonesia
Shinta Nuriyah Mahbubiyah Royani Universitas Negeri Malang, Indonesia https://orcid.org/0009-0003-8219-1313
Bintan Nuril Irvaniyah Universitas Negeri Malang, Indonesia
Natasya Mustika Elvina Rossa Junior High School Antartika, Indonesia
Anggita Meilina Putri Universitas Negeri Malang, Indonesia
Khoirun Nisa' National Dong Hwa University, Taiwan

DOI:

https://doi.org/10.21067/mpej.v10i1.13156

Keywords:

Conceptual understanding, Instrument validation, RASCH model, Temperature and heat

Abstract

This study aims to analyze the quality of a concept understanding test instrument on temperature and heat using the Rasch Model approach, focusing on validity, reliability, and item difficulty level. This research employed a descriptive method with a quantitative approach. The sample was selected through simple random sampling and involved 75 students of class XI at Senior High School. The research instrument consisted of 15 multiple-choice items covering concepts related to temperature and heat. Data were analyzed using the Rasch Model assisted by Ministep software to examine item fit, person reliability, item reliability, and the distribution of item difficulty. The results showed that all 15 items were valid based on the INFIT and OUTFIT criteria of MNSQ and ZSTD values. The person reliability was categorized as poor, whereas the item reliability was categorized as very good. The item difficulty level was distributed into four categories, consisting of 4 easy items, 2 moderate items, 6 difficult items, and 3 very difficult items. These findings indicate that the Rasch Model is effective in identifying the validity, reliability, and difficulty distribution of test items. The study implies that the developed instrument can be used by physics teachers and researchers as an empirical basis for evaluating students’ conceptual understanding of temperature and heat. However, the low person reliability suggests the need for further refinement by involving a more heterogeneous sample and developing additional items with a broader range of difficulty levels to improve the instrument’s ability to distinguish students’ conceptual understanding.

Downloads

Download data is not yet available.

References

Akbari, A. (2025). The Rasch analysis of item response theory: An untouched area in evaluating student academic translations. SKASE Journal of Translation and Interpretation, 18(1), 50–77.

Alnahdi, A. H., Alsubiheen, A. M., & Aldaihan, M. M. (2025). Rasch measurement model supports the unidimensionality and internal structure of the Arabic Oswestry Disability Index. Journal of Clinical Medicine, 14(4), 1259. https://doi.org/10.3390/jcm14041259

Antonio, R. P., & Prudente, M. S. (2022). Effectiveness of metacognitive instruction on students' science learning achievement: A meta-analysis. International Journal on Studies in Education, 4(1), 1–20.

Arikunto, S. (2017). Pengembangan instrumen penelitian dan penilaian program. Pustaka Pelajar.

Avinç, E., & Doğan, F. (2024). Digital literacy scale: Validity and reliability study with the Rasch model. Education and Information Technologies, 29(17), 22895–22941. https://doi.org/10.1007/s10639-024-12657-2

Bond, T. G., & Fox, C. M. (2013). Applying the Rasch model: Fundamental measurement in the human sciences (3rd ed.). Routledge.

Christensen, K. S., & Ammentorp, J. (2024). Rasch analysis of the self-efficacy (SE-12) questionnaire measuring clinical communication skills. PEC Innovation, 4, 100296. https://doi.org/10.1016/j.pecinn.2024.100296

Dewi, S. Z., & Ibrahim, T. (2019). Pentingnya pemahaman konsep untuk mengatasi miskonsepsi dalam materi belajar IPA di sekolah dasar. Jurnal Pendidikan Universitas Garut, 13(1), 130–136.

Discipulo, L. G., & Bautista, R. G. (2022). Students’ cognitive and metacognitive learning strategies towards hands-on science. International Journal of Evaluation and Research in Education, 11(2), 882–891. https://doi.org/10.11591/ijere.v11i2.21919

Hamzah, H. (2017). Learning about A-level physics students' understanding of particle physics using concept mapping. Physics Education, 52(5), Article 054002. https://doi.org/10.1088/1361-6552/aa7f3c

Hamzah, F. M., Rashid, M. N. A., Rahman, M. N. A., & Rasul, M. S. (2022). Evaluating the validity and reliability of authentic learning instruments using Rasch model. International Journal of Global Optimization and Its Application, 1(3), 182–189. https://doi.org/10.56225/ijgoia.v1i3.24

Handayani, S. (2022). Analisis UAS biologi kelas X dengan teori tes klasik dan item response theory (Rasch model). Bio-Pedagogi, 11(2), 76–84. https://doi.org/10.20961/bio-pedagogi.v11i2.63177

Hermita, N., Sakinah, S., Wijaya, T. T., Vebrianto, R., Alim, J. A., Putra, Z. H., Yulianti, E., & Jihe, C. (2021). Item analysis of heat transfer concept using Rasch model in elementary school. Journal of Physics: Conference Series, 2049(1), Article 012058. https://doi.org/10.1088/1742-6596/2049/1/012058

Isnawati, I., Sriyati, S., Agustin, R. R., Supriyadi, S., Kasi, Y. F., & Ismail, I. (2024). Analysis of question difficulty levels based on science process skills indicators using the Rasch model. Tadris: Jurnal Keguruan dan Ilmu Tarbiyah, 9(1), 31–41. https://doi.org/10.24042/tadris.v9i1.19374

Kementerian Pendidikan dan Kebudayaan Republik Indonesia. (2016). Peraturan Menteri Pendidikan dan Kebudayaan Republik Indonesia Nomor 20 Tahun 2016 tentang Standar Kompetensi Lulusan Pendidikan Dasar dan Menengah. Kemendikbud.

L'Boy, D., & Khan, R. N. (2023). A Rasch-model-based hierarchical framework for statistical literacy and learning. International Journal of Mathematical Education in Science and Technology, 54(9), 1874–1887. https://doi.org/10.1080/0020739X.2022.2060166

Lestari, A. S., & Samsudin, A. (2020). Using Rasch model analysis to analyze students’ scientific literacy on heat and temperature. In Proceedings of the 7th Mathematics, Science, and Computer Science Education International Seminar (pp. xx–xx).

Lestari, K. E., & Yudhanegara, M. R. (2017). Analisis kemampuan representasi matematis mahasiswa pada mata kuliah geometri transformasi berdasarkan latar belakang pendidikan menengah. Jurnal Matematika Integratif, 13(1), 29–39. https://doi.org/10.24198/jmi.v13i1.11410

Lichtenberger, A., Hofer, S. I., Stern, E., & Vaterlaus, A. (2025). Enhanced conceptual understanding through formative assessment: Results of a randomized controlled intervention study in physics classes. Educational Assessment, Evaluation and Accountability, 37(1), 5–33. https://doi.org/10.1007/s11092-024-09458-5

Metsämuuronen, J. (2023). Seeking the real item difficulty: Bias-corrected item difficulty and some consequences in Rasch and IRT modeling. Behaviormetrika, 50(1), 121–154. https://doi.org/10.1007/s41237-022-00192-7

Müller, M. (2020). Item fit statistics for Rasch analysis: Can we trust them? Journal of Statistical Distributions and Applications, 7(1), Article 5. https://doi.org/10.1186/s40488-020-00108-7

Muna, I. A. (2017). Model pembelajaran POE (predict–observe–explain) dalam meningkatkan pemahaman konsep dan keterampilan proses IPA. El-Wasathiya: Jurnal Studi Agama, 5(1), 73–92.

Hamzah, F. M., Rashid, M. N. A., Rahman, M. N. A., & Rasul, M. S. (2022). Evaluating the validity and reliability of authentic learning instruments using Rasch model. International Journal of Global Optimization and Its Application, 1(3), 182–189. https://doi.org/10.56225/ijgoia.v1i3.24

Papini, N., Kang, M., Ryu, S., Griese, E., Wingert, T., & Herrmann, S. (2021). Rasch calibration of the 25-item Connor–Davidson Resilience Scale. Journal of Health Psychology, 26(11), 1976–1987. https://doi.org/10.1177/1359105320909876

Putranta, H., & Supahar, S. (2019). Development of physics-tier tests (PysTT) to measure students' conceptual understanding and creative thinking skills: A qualitative synthesis. Journal for the Education of Gifted Young Scientists, 7(3), 747–775. https://doi.org/10.17478/jegys.587203

Putri, A. P., & Sayono, J. (2026). Evaluation of item quality: Analysis of difficulty level and distinction power with quantitative methods. Journal of Educational Sciences, 10(1), 317–330.

Rahmayani, A., Tiurlina, T., & Alfarisa, F. (2022). Analisis kualitas butir soal ulangan harian matematika di kelas IV MI Al-Islamiyah menggunakan Rasch model. Jurnal Perseda, 5(3), 170–177.

Royani, S. N. M., Sutopo, S., Hidayat, A., & Parno, P. (2025). Research trends in physics learning strategies: A systematic literature review addressing students' conceptual understanding difficulties in kinematics. Jurnal Penelitian Pendidikan IPA, 11(3), 1–10.

Royani, S., Artanti, K., Putri, R., & Parno, P. (2025). Analysis of item difficulty and student misconceptions on temperature and heat using a two-tier diagnostic test. Jurnal Ilmiah Pendidikan Fisika, 9(1), 144–154. https://doi.org/10.20527/jipf.v9i1.

Safitri, F., Rusdiana, D., Samsudin, A., & Widiyatmoko, A. (2025). Development of understanding test instruments for grade 7 junior high school students on temperature, heat and expansion topics. Jurnal Penelitian Pendidikan IPA, 11(1), 395–404. https://doi.org/10.29303/jppipa.v11i1.

Shidik, M. A. (2020). Hubungan antara motivasi belajar dengan pemahaman konsep fisika peserta didik MAN Baraka. Jurnal Kumparan Fisika, 3(2), 91–98. https://doi.org/10.33369/jkf.3.2.91-98

Setyaningrum, N., & Fadli, F. (2023). Cognitive development analysis of C4, C5, and C6 in early childhood education students and their implications for teacher efforts at Raudhatul Athfal. Proceeding International Conference on Religion, Science, and Education, 2, 487–496.

Subando, J., Astoko, D. B., Amin, L. H., Budiharjo, & Embong, R. (2025). A multidimensional item response theory approach in the item analysis of Arabic language tests in Madrasah Aliyah. Jurnal Penelitian dan Evaluasi Pendidikan, 29(2), 271–284.

Sumintono, B., & Widhiarso, W. (2015). Aplikasi pemodelan Rasch pada assessment pendidikan. Trim Komunikata.

Suparno, P. (2005). Miskonsepsi dan perubahan konsep dalam pendidikan fisika. Grasindo.

Rismawati, T. A., & Pujiastuti, H. (2020). Pengaruh model Search Solve Create and Share (SSCS) terhadap kemampuan pemahaman konsep matematis. Jurnal Kajian Pendidikan Matematika, 5(2), 183–190. https://doi.org/10.30998/jkpm.v5i2.6332

Tennant, A., McKenna, S. P., & Hagell, P. (2004). Application of Rasch analysis in the development and application of quality of life instruments. Value in Health, 7(Suppl. 1), S22–S26. https://doi.org/10.1111/j.1524-4733.2004.7s106.x

Yang, J., Reheman, Z., Liu, Y., & Zhao, S. (2023). Applying Rasch analysis in refinement and validation of interpersonal skills measure for gifted children. Frontiers in Psychology, 14, Article 1236640. https://doi.org/10.3389/fpsyg.2023.1236640

Yulianto, A., & Widodo, A. (2020). Disclosure of difficulty distribution of HOTS-based test questions through Rasch modeling. Indonesian Journal of Primary Education, 4(2), 197–203. https://doi.org/10.17509/ijpe.v4i2.26691

Yusuf, I., & Widyaningsih, S. W. (2018). Profil kemampuan mahasiswa dalam menyelesaikan soal HOTS di Jurusan Pendidikan Fisika Universitas Papua. Jurnal Komunikasi Pendidikan, 2(1), 42–49. https://doi.org/10.32585/jkp.v2i1.63