Analysis of critical thinking test instruments on the Light Waves concept (CT-LiWa): Rasch model

Raldy Aditya; Endi Suhendi; Lina Aviyanti; Abu Nawas; Heni Rusnayati

doi:10.21067/mpej.v9i2.11662

Authors

Raldy Aditya Universitas Pendidikan Indonesia, Indonesia
Endi Suhendi Universitas Pendidikan Indonesia, Indonesia https://orcid.org/0000-0001-8256-8537
Lina Aviyanti Universitas Pendidikan Indonesia, Indonesia https://orcid.org/0000-0003-1152-9703
Abu Nawas The University of Adelaide, Australia https://orcid.org/0000-0003-0365-1020
Heni Rusnayati Universitas Pendidikan Indonesia, Indonesia https://orcid.org/0000-0002-8530-0744

DOI:

https://doi.org/10.21067/mpej.v9i2.11662

Keywords:

Critical thinking, Light waves, Rasch model

Abstract

This research aims to develop an instrument that measures students' critical thinking skills in physics learning to test its validity and reliability using the Rasch Model. The instrument is called the Critical Thinking Test on The Light Waves Concept (CT-LiWa). The novelty of this research is to create a new instrument that is applied to the concept of light waves. The target of this instrument is high school students in one of the provinces in West Java. This research method uses the ADDIE model involving 72 high school students from West Java, Indonesia. The instrument used is a validation sheet and the CT-LiWa consists of 15 items in the form of descriptions. The trial results data were analyzed using the Rasch Model assisted by the Winstep program to obtain the quality of the instrument, namely reliability, validity, and level of difficulty. The results of the analysis show that the CT-LiWa test is not reliable but has a good validity of 27.6% in the fulfilled category with varying levels of difficulty. Therefore, the development of the CT-LiWa test can be implemented as an instruent to measure students' critical thinking skills on the concept of light waves.

Downloads

Download data is not yet available.

References

Adams, D., Sumintono, B., Mohamed, A., & Noor, N. S. M. (2018). E-learning readiness among students of diverse backgrounds in a leading Malaysian higher education institution. Malaysian Journal of Learning and Instruction, 15(2), 227–256. https://doi.org/10.32890/mjli2018.15.2.9

Aditya, R., Suhendi, E., & Samsudin, A. (2023). Exploring high school students’ critical thinking skills using active learning: A systematic literature review. International Journal of Engineering Research & Technology (IJERT), 12(6), 1–8. https://www.ijert.org/exploring-high-school-students-critical-thinking-skills-using-active-learning-a-systematic-literature-review IJERT

Ariani, T. (2020). Analysis of students’ critical thinking skills in physics problems. Kasuari: Physics Education Journal (KPEJ), 3(1), 1–17. https://doi.org/10.37891/kpej.v3i1.119 ResearchGate+1

Arsad, N., Kamal, N., Ayob, A., Sarbani, N., Tsuey, C. S., Misran, N., & Husain, H. (2013). Rasch model analysis on the effectiveness of early evaluation questions as a benchmark for new students’ ability. International Education Studies, 6(6), 185–190. https://doi.org/10.5539/ies.v6n6p185

Aulia, E. (2022). Effects of 21st century learning on the development of critical thinking, creativity, communication, and collaboration skills. Journal of Nonformal Education, 8(1), 46–53. https://doi.org/10.15294/jne.v8i1.33334

Bhakti, Y. B., Arthur, R., & Supriyati, Y. (2023). Development of an assessment instrument for critical thinking skills in physics: A systematic literature review. Journal of Physics: Conference Series, 2596(1), Article 012067. https://doi.org/10.1088/1742-6596/2596/1/012067

Clark, L. A., & Watson, D. (2019). Constructing validity: New developments in creating objective measuring instruments. Psychological Assessment, 31(12), 1412–1427. https://doi.org/10.1037/pas0000626

Ennis, R. H. (1989). Critical thinking and subject specificity: Clarification and needed research. Educational Researcher, 18(3), 4–10. https://doi.org/10.3102/0013189X018003004

Ennis, R. H. (1993). Critical thinking assessment. Theory Into Practice, 32(3), 179–186. https://doi.org/10.1080/00405849309543594

Ennis, R. H. (2003). Critical thinking assessment. In D. Fasko (Ed.), Critical thinking and reasoning: Current research, theory and practice (pp. —-). Hampton Press.

Facione, P. A. (1990a). The California Critical Thinking Skills Test: College level. California Academic Press.

Facione, P. A. (1990b). Critical thinking: A statement of expert consensus for purposes of educational assessment and instruction (The Delphi Report). American Philosophical Association. https://www.insightassessment.com

Fisher, W. P. (2007). Rating scale instrument quality criteria. Rasch Measurement Transactions, 21(1), 1095.

Gunawan, G., Harjono, A., Herayanti, L., & Husein, S. (2019). Problem-based learning approach with supported interactive multimedia in physics course: Its effects on critical thinking disposition. Journal for the Education of Gifted Young Scientists, 7(4), 1075–1089. https://doi.org/10.17478/jegys.627162

Hagell, P. (2014). Testing rating scale unidimensionality using the principal component analysis (PCA)/t-test protocol with the Rasch model: The primacy of theory over statistics. Open Journal of Statistics, 4(6), 456–465. https://doi.org/10.4236/ojs.2014.46044

Halimatun Sa, L., Siahaan, P., Suhendi, E., Samsudin, A., Aminudin, A. H., Rais, A., Sari, I., & Rachmadtullah, R. (2020). Critical Thinking Instrument Test (CTIT): Developing and analyzing Sundanese students’ critical thinking skills on physics concepts using Rasch analysis. International Journal of Psychosocial Rehabilitation, 24(8), 3276–3290. https://doi.org/10.37200/IJPR/V24I8/PR281423

Halpern, D. F. (2003). The “how” and “why” of critical thinking assessment. In D. Fasko (Ed.), Critical thinking and reasoning: Current research, theory, and practice (pp. 1–23). Hampton Press.

Halpern, D. F. (2007). Halpern Critical Thinking Assessment using everyday situations: Background and scoring standards. Claremont McKenna College.

Halpern, D. F. (2010). Halpern Critical Thinking Assessment: Manual. Schuhfried GmbH.

Halpern, D. F. (2014). Thought and knowledge: An introduction to critical thinking (5th ed.). Psychology Press.

Jamil, M., Abdul Hafeez, F., & Muhammad, N. (2024). Critical thinking development for 21st century: Analysis of physics curriculum. Journal of Social & Organizational Matters, 2(3), 1–10. https://doi.org/10.56976/jsom.v3

Kealey, B. T., Holland, J., & Watson, M. (2005). Preliminary evidence on the association between critical thinking and performance in principles of accounting. Issues in Accounting Education, 20(1), 33–49. https://doi.org/10.2308/iace.2005.20.1.33

Ku, K. Y. L. (2009). Assessing students’ critical thinking performance: Urging for measurements using multi-response format. Thinking Skills and Creativity, 4(1), 70–76. https://doi.org/10.1016/j.tsc.2009.02.001

Lawshe, C. H. (1975). A quantitative approach to content validity. Personnel Psychology, 28(4), 563–575. https://doi.org/10.1111/j.1744-6570.1975.tb01393.x

Lawson, A. E. (2004). The nature and development of scientific reasoning: A synthetic view. International Journal of Science and Mathematics Education, 2(3), 307–338. https://doi.org/10.1007/s10763-004-3224-2

Linacre, J. M. (1994). Sample size and item calibration stability. Rasch Measurement Transactions, 7(4), 328. https://www.researchgate.net/publication/235361463

Lin, S.-S. (2014). Science and non-science undergraduate students’ critical thinking and argumentation performance in reading a science news report. International Journal of Science and Mathematics Education, 12(5), 1023–1046. https://doi.org/10.1007/s10763-013-9442-5

Norris, S. P., & Ennis, R. H. (1989). Evaluating critical thinking. Midwest Publications.

Park, M., & Liu, X. (2021). An investigation of item difficulties in energy aspects across biology, chemistry, environmental science, and physics. Research in Science Education, 51(1), 43–60. https://doi.org/10.1007/s11165-019-9819-y

Pascarella, E. T., & Terenzini, P. T. (2005). How college affects students: A third decade of research (Vol. 2). Jossey-Bass.

Putranta, H., & Wilujeng, I. (2019). Physics learning by PhET simulation-assisted using problem-based learning (PBL) model to improve students’ critical thinking skills in work and energy chapters in MAN 3 Sleman. Asia-Pacific Forum on Science Learning and Teaching, 20(1), Article 7. https://www.eduhk.hk/apfslt/download/v20_issue1_files/putranta.pdf

Raof, S. A., Musta’amal, A. H., Zamzuri, F. K., & Salleh, M. H. (2021). Validity and reliability of students’ perceptions on OBE approach in Malaysian VC using Rasch model. Journal of Innovation in Educational and Cultural Research, 2(2), 44–50. https://doi.org/10.46843/jiecr.v2i2.30

Recker, J. (2021). Scientific research in information systems: A beginner’s guide (2nd ed.). Springer. https://doi.org/10.1007/978-3-030-85432-4

Samsudin, A., Cahyani, P. B., Purwanto, Rusdiana, D., Efendi, R., Aminudin, A. H., & Coştu, B. (2021). Development of a multitier open-ended work and energy instrument (MOWEI) using Rasch analysis to identify students’ misconceptions. Cypriot Journal of Educational Sciences, 16(1), 16–31. https://doi.org/10.18844/cjes.v16i1.5504

Samsudin, A., Fratiwi, N. J., Ramalis, T. R., Aminudin, A. H., Coştu, B., & Nurtanto, M. (2020). Using Rasch analysis to develop multi-representation tier instrument on Newton’s law (MOTION). International Journal of Psychosocial Rehabilitation, 24(6), 8542–8556. https://doi.org/10.37200/IJPR/V24I6/PR260875

Sapnas, K. G., & Zeller, R. A. (2002). Minimizing sample size when using exploratory factor analysis for measurement. Journal of Nursing Measurement, 10(2), 135–154. https://doi.org/10.1891/jnum.10.2.135.52552

Saputra, M. D., Joyoatmojo, S., Wardani, D. K., & Sangka, K. B. (2019). Developing critical-thinking skills through the collaboration of jigsaw model with problem-based learning model. International Journal of Instruction, 12(1), 1077–1094. https://doi.org/10.29333/iji.2019.12169a

Sumarni, W., Supardi, K. I., & Widiarti, N. (2018). Development of assessment instruments to measure critical thinking skills. IOP Conference Series: Materials Science and Engineering, 349(1), 012066. https://doi.org/10.1088/1757-899X/349/1/012066

Sumintono, B. (2018). Rasch model measurements as tools in assessment for learning. In Proceedings of the 1st International Conference on Education Innovation (ICEI 2017) (pp. 11–15). Atlantis Press. https://doi.org/10.2991/icei-17.2018.11

Sumintono, B., & Widhiarso, W. (2014). Aplikasi model Rasch untuk penelitian ilmu-ilmu sosial (edisi revisi; B. Trim, Ed.). Trim Komunikata Publishing House.

Szabo, Z. K., Körtesi, P., Guncaga, J., Szabo, D., & Neag, R. (2020). Examples of problem-solving strategies in mathematics education supporting the sustainability of 21st-century skills. Sustainability, 12(23), 10113. https://doi.org/10.3390/su122310113

Taube, K. T. (1997). Critical thinking ability and disposition as factors of performance on a written critical thinking test. The Journal of General Education, 46(2), 129–164. https://doi.org/10.1353/jge.2007.0021

Teo, P. (2019). Teaching for the 21st century: A case for dialogic pedagogy. Learning, Culture and Social Interaction, 21, 170–178. https://doi.org/10.1016/j.lcsi.2019.03.009

Tiruneh, D. T., De Cock, M., Weldeslassie, A. G., Elen, J., & Janssen, R. (2017). Measuring critical thinking in physics: Development and validation of a critical thinking test in electricity and magnetism. International Journal of Science and Mathematics Education, 15(4), 663–682. https://doi.org/10.1007/s10763-016-9723-0

Wartono, W., Hudha, M. N., & Batlolona, J. R. (2018). How are the physics critical thinking skills of the students taught by using inquiry-discovery through empirical and theoretical overview? Eurasia Journal of Mathematics, Science and Technology Education, 14(2), 691–697. https://doi.org/10.12973/ejmste/80632

Watson, G., & Glaser, E. M. (1980). Watson–Glaser critical thinking appraisal. Psychological Corporation.

Zakaria, N. H., Phang, F. A., & Pusppanathan, J. (2019). Physics on the go: A mobile computer-based physics laboratory for learning forces and motion. International Journal of Emerging Technologies in Learning (iJET), 14(24), 167–183. https://doi.org/10.3991/ijet.v14i24.12063