Developing authentic assessment instrument based on multiple representations to measure students' critical thinking skills

Main Article Content

Muhammad Minan Chusni
Suherman Suherman

Abstract

The purpose of this study was to produce an instruments test used to measure students' critical thinking skills in natural science learning. This research uses a 4-D development model (define, design, develop, and disseminate) involving 118 students at the develop stage and 60 students at the disseminate stage. The instrument developed was an essay test based on multiple representations. Validity was proved by using CVI, and reliability was estimated by using the Item Response Theory. The results showed that the instrument had a very good foreign exchange value. This is reflected in the Aiken V scores on the aspects of substance, construct, language and appearance, respectively, about 1.00, 1.00, 1.00, and 1.00. According to Rasch analysis, the instruments has meet the assumption test for 14 items which is unidimensional, the local independence assumption test, and the parameter invariance assumption test. According to OUTFIT MNSQ Value, the items are fit with PCM 1-PL which functions normally in making measurements. Reliability estimated for items shows a very high consistency of measurement of 0.97 and for person shows a high consistency of measurement of 0.86. The results of the student's CTS measurement showed that the average score was 65.50, with a distribution of high, medium, and low abilities, respectively, about 16.67%, 63.33%, and 20.00%. Thus, according to these results, an authentic assessment based on multiple representations is suitable to measure students' critical thinking skills.

Article Details

How to Cite
Chusni, M. M., & Suherman, S. (2021). Developing authentic assessment instrument based on multiple representations to measure students’ critical thinking skills. Momentum: Physics Education Journal, 5(2), 194-208. https://doi.org/10.21067/mpej.v5i2.5790
Section
Articles

References

Abdurrahman, A., Setyaningsih, C. A., & Jalmo, T. (2019). Implementating multiple representation-based worksheet to develop critical thinking skills. Journal of Turkish Science Education, 16(1), 138–155. https://doi.org/10.12973/tused.10271a

Amalia, S., & Wuryandani, W. (2020). Socio-cultural based learning module for critical thinking ability in elementary school: systematic search. Acta Educationis Generalis, 10(2), 180–187. https://doi.org/10.2478/atd-2020-0017

Arends, R. I., & Kilcher, A. (2010). Teaching for student learning: Becoming an accomplished teacher. In Teaching for Student Learning: Becoming an Accomplished Teacher. https://doi.org/10.4324/9780203866771

Boone, W. J., & Noltemeyer, A. (2017). Rasch analysis: A primer for school psychology researchers and practitioners. Cogent Education, 4(1), 1–13. https://doi.org/10.1080/2331186X.2017.1416898

Boone, W. J., Townsend, J. S., & Staver, J. (2011). Using Rasch theory to guide the practice of survey development and survey data analysis in science education and to inform science reform efforts: An exemplar utilizing STEBI self-efficacy data. Science Education, 95(2), 258–280. https://doi.org/10.1002/sce.20413

Boone, W. J., Yale, M. S., & Staver, J. R. (2014). Rasch analysis in the human sciences. Springer. https://doi.org/10.1007/978-94-007-6857-4

Brookhart, S. M., & Nitko, A. J. (2019). Educational assessment of students (8th ed.). Pearson.

Burhanuddin. (2015). Evaluasi keterlaksanaan penilaian otentik (authentic assessment) pada pembelajaran IPA SMP Negeri di Pasangkayu Kabupaten Mamuju Utara [Universitas Negeri Yogyakarta]. https://eprints.uny.ac.id/25593/

Chan, S. W., Ismail, Z., & Sumintono, B. (2014). A Rasch model analysis on secondary students’ statistical reasoning ability in descriptive statistics. Procedia - Social and Behavioral Sciences, 129, 133–139. https://doi.org/10.1016/j.sbspro.2014.03.658

Chusni, M. M., Saputro, S., Raharjo, S. B., & Suranto, S. (2021). Student’s critical thinking skills through discovery learning model using e-learning on environmental change subject matter. European Journal of Educational Research, 10(3), 1123–1135. https://doi.org/10.12973/eu-jer.10.3.1123

Chusni, M. M., Saputro, S., Suranto, S., & Rahardjo, S. B. (2020). Review of critical thinking skill in indonesia: Preparation of the 21st century learner. Journal of Critical Reviews, 7(09), 1230–1235. https://doi.org/10.31838/jcr.07.09.223

Doleck, T., Bazelais, P., Lemay, D. J., Saxena, A., & Basnet, R. B. (2017). Algorithmic thinking, cooperativity, creativity, critical thinking, and problem solving: exploring the relationship between computational thinking skills and academic performance. Journal of Computers in Education, 4(4), 355–369. https://doi.org/10.1007/s40692-017-0090-9

Fariña, P., González, J., & San Martín, E. (2019). The use of an identifiability-based strategy for the interpretation of parameters in the 1PL-G and rasch models. Psychometrika, 84(2), 511–528. https://doi.org/10.1007/s11336-018-09659-w

Fonna, M., & Mursalin, M. (2018). Role of self-efficacy toward students’ achievement in mathematical multiple representation ability (MMRA). Jurnal Ilmiah Peuradeun, 6(1), 31. https://doi.org/10.26811/peuradeun.v6i1.174

Fu, J., & Feng, Y. (2018). A comparison of score aggregation methods for unidimensional tests on different dimensions. ETS Research Report Series, 2018(1), 1–16. https://doi.org/10.1002/ets2.12194

Gardner, A. K., & Dunkin, B. J. (2018). Evaluation of validity evidence for personality, emotional intelligence, and situational judgment tests to identify successful residents. JAMA Surgery, 153(5), 409–415. https://doi.org/10.1001/jamasurg.2017.5013

Gebre, E. (2018). Learning with multiple representations: Infographics as cognitive tools for authentic learning in science literacy. Canadian Journal of Learning and Technology, 44(1), 1–24. https://doi.org/10.21432/cjlt27572

Hambleton, R. K., & Swaminathan, H. (1985). Item response theory. Springer Netherlands. https://doi.org/10.1007/978-94-017-1988-9

Hopfenbeck, T. N., Lenkeit, J., El Masri, Y., Cantrell, K., Ryan, J., & Baird, J.-A. (2018). Lessons learned from PISA: A systematic review of peer-reviewed articles on the programme for international student assessment. Scandinavian Journal of Educational Research, 62(3), 333–353. https://doi.org/10.1080/00313831.2016.1258726

Kartianom, K., & Ndayizeye, O. (2017). What‘s wrong with the Asian and African Students’ mathematics learning achievement? The multilevel PISA 2015 data analysis for Indonesia, Japan, and Algeria. Jurnal Riset Pendidikan Matematika, 4(2), 200–210. https://doi.org/10.21831/jrpm.v4i2.16931

Kristidhika, D. C., Cendana, W., Felix-Otuorimuo, I., & Müller, C. (2020). Contextual teaching and learning to improve conceptual understanding of primary students. Teacher in Educational Research, 2(2), 71–78. https://doi.org/10.33292/ter.v2i2.84

Kvale, S. E. (1989). Issues of validity in qualitative research. Studentlitteratur.

Lawshe, C. H. (1975). A quantitative approach to content validity. Personnel Psychology, 28(4), 563–575. http://caepnet.org/~/media/Files/caep/knowledge-center/lawshe-content-validity.pdf

Lester, P. E., Inman, D., & Bishop, L. K. (2014). Handbook of tests and measurement in education and the social sciences. Rowman & Littlefield.

Linacre, J. M. (2012). A user guide to Winsteps Ministep Rasch model computer programs: Program manual 3.75.0.

Maba, W. (2017). Teacher’s perception on the implementation of the assessment process in 2013 curriculum. International Journal of Social Sciences and Humanities, 1(2), 1–9. https://doi.org/10.29332/ijssh.v1n2.26

Menteri Pendidikan dan Kebudayaan Republik Indonesia. (2016). Peraturan Menteri Pendidikan dan Kebudayaan Republik Indonesia Nomor 23 Tahun 2016 tentang standar penilaian pendidikan. Kementerian Pendidikan dan Kebudayaan Republik Indonesia.

Menteri Pendidikan dan Kebudayaan Republik Indonesia. (2019). Peraturan Menteri Pendidikan dan Kebudayaan Republik Indonesia Nomor 43 Tahun 2019 tentang penyelenggaraan ujian yang diselenggarakan satuan pendidikan dan ujian nasional. Kementerian Pendidikan dan Kebudayaan Republik Indonesia.

Moore, T. J., Brophy, S. P., Tank, K. M., Lopez, R. D., Johnston, A. C., Hynes, M. M., & Gajdzik, E. (2020). Multiple representations in computational thinking tasks: A clinical study of second-grade students. Journal of Science Education and Technology, 29(1), 19–34. https://doi.org/10.1007/s10956-020-09812-0

Mujib, N. R., Toenlioe, A. J. ., & Praherdhiono, H. (2018). Analisis butir soal ujian nasional IPA SD/MI tahun 2015 sampai 2017 berdasarkan taksonomi Bloom. Jktp, 1(2), 149–158. http://journal2.um.ac.id/index.php/jktp/article/view/3731

Mutia, N. B., & Prasetyo, Z. K. (2018). The effectiveness of students’ worksheet based on multiple representations to increase creative thinking skills. Journal of Education and Learning (EduLearn), 12(4), 631–637. https://doi.org/10.11591/edulearn.v12i4.8487

Namdar, B., & Shen, J. (2016). Intersection of argumentation and the use of multiple representations in the context of socioscientific issues. International Journal of Science Education, 38(7), 1100–1132. https://doi.org/10.1080/09500693.2016.1183265

Ngin, C. S. (2018). Examining a teacher’s use of multiple representations in the teaching of percentages : A commognitive perspective. Proceedings of the 41st Annual Conference of the Mathematics Education Research Group of Australasia, 591–598.

Nugrahanto, S., & Zuchdi, D. (2019). Indonesia PISA result and impact on the reading learning program in Indonesia. Proceedings of the International Conference on Interdisciplinary Language, Literature and Education (ICILLE 2018), 297(Icille 2018), 373–377. https://doi.org/10.2991/icille-18.2019.77

Paul, R., Elder, L., & Bartell, T. (1997). A brief history of the idea of critical thinking. California Commission on Teacher Credentialing. http://www.criticalthinking.org/pages/a-brief-history-of-the-idea-of-critical-thinking/408

Permatasari, A. K., Istiyono, E., & Kuswanto, H. (2019). Developing assessment instrument to measure physics problem solving skills for mirror topic. International Journal of Educational Research Review, 358–366. https://doi.org/10.24331/ijere.573872

Prahani, B. K., Limatahu, I., W.W, S., Yuanita, L., & Nur, M. (2016). Effectiveness of physics learning material through guided inquiry model to improve student’s problem solving skills based on multiple representation. International Journal of Education and Research, 4(12), 231–242. https://www.ijern.com/journal/2016/December-2016/17.pdf

Rachman, D., Soviyah, S., Fajaruddin, S., & Pratama, R. A. (2020). Reading engagement, achievement and learning experiences through kahoot. LingTera, 7(2). https://doi.org/10.21831/lt.v7i2.38457

Suprapto, N. (2016). What should educational reform in Indonesia look like? - Learning from the PISA science scores of East-Asian countries and Singapore. Asia-Pacific Forum on Science Learning and Teaching, 17(2), 1–22.

Susilaningsih, E., Supartono, S., Kristanto, T., Sariana, E., Azizah, P., & Natasukma, M. (2019). The effectiveness of multiple representation oriented learning material with project based learning to improve students’ chemistry learning outcomes. Proceedings of the 6th International Conference on Educational Research and Innovation (ICERI 2018), 330(Iceri 2018), 87–90. https://doi.org/10.2991/iceri-18.2019.18

Wijayanti, M. D., Rahardjo, S. B., Saputro, S., & Mulyani, S. (2019). Item analysis of critical thinking skills instrument to measure effectiveness of scientific group inquiry learning (SGIL) model. Jurnal Pendidikan IPA Indonesia, 8(4), 538–546. https://doi.org/10.15294/jpii.v8i4.20794

Wright, B. D., & Stone, M. H. (1988). Reliability in Rasch measurement. In Research Memorandum No. 53. MESA.

Yanti, H., Distrik, I. W., & Rosidin, U. (2019). The effectiveness of students’ worksheets based on multi-representation in improving students’ metacognition skills in static electricity. Journal of Physics: Conference Series, 1155(1), 012083. https://doi.org/10.1088/1742-6596/1155/1/012083

Yanto, F. (2019). Development of problem-based student worksheet with authentic assessment to improve student’s physics problem solving ability. Journal of Physics: Conference Series, 1185, 012075. https://doi.org/10.1088/1742-6596/1185/1/012075

Yusrizal, Y. (2016). Analysis of difficulty level of physics national examination’s questions. Jurnal Pendidikan IPA Indonesia, 5(1), 140–149. https://doi.org/10.15294/jpii.v5i1.5803