Development of augmented reality integrated physics e-worksheet to improve students' problem-solving skills

: The STEM approach through the use of augmented reality can provide visualization to students in online learning. The purposes of this research are (1) to develop Physics E-Worksheet based on STEM using augmented reality system, and (2) to improve the problem-solving skills of high school students on Momentum and Impulse material. The type of research used is research and development (R&D) with a 4D model (define, design, develop and disseminate). This study involved 24 students as the field trial class and 31 student of class X as operational field class. The results of this study are: (1) The AR Physics E-Worksheet is valid and reliable to be used in the physical learning process of Momentum and Impulse material based on the results of validation analysis using the average value of four-scale with a score of 3.73 in very good criteria. (2) The results of the Paired T-Test analysis showed the value of Sig. 0.00, so that there is an effect of using AR Physics E-Worksheet on improving problem-solving skills of high school students. The score of Normalized Gain obtained 0.77 in high category improvement. The results showed that the AR Physics E-Worksheet was effective in improving problem solving skills so that it could be used as a learning medium for the topic of momentum and impulse by class X science students.


Introduction
STEM-based learning approach is applied as a basis in the learning process of several scientific fields. STEM learning utilizes the integration of Science, Technology, Engineering and Mathematics to improve the potential of students (Nugroho et al., 2019). In this era, STEM-based learning approach is also used in the online education system. STEM does not only have an influence on fields of study related to science and mathematics, but also affects other learning broadly. The use of STEM in learning needs to be adjusted to the learning content so it is able to provide an increase in cognitive and non-cognitive outcomes (Adams, 2021).
Physics is related to the rules of natural and the behavior of the universe expressed in mathematical equations (Fidan & Tuncel, 2021). Physics studies natural phenomena related to theories, concepts, laws, formulas and their application through the scientific process starting from investigation activities to discoveries, that create technological innovations (Siswono, 2017). However, students encounter problems when they have to analyze the problem-solving process and convert the results of their discussions into a plan that can be implemented to achieve a problem solution (Chang et al., 2017). Students found difficulties when they have to do so many physics concepts taught in secondary schools. The concept is an abstract concept of nature so it requires effort in describing the concept. Conceptual descriptions that are carried out orally are considered boring and less understandable by students. The low concept that students understand makes it difficult to solve physics problems (Ogunkola & Samuel, 2011).
Problem is interpreted as a difficulty in an organized systematic situation that encourages the curiosity of students to solve that difficulty. Analysis of the problems that exist of perceiving problems, solving problems, awareness of problems, and strategies for obtaining solutions to the problems, will lead to new knowledge understanding for students (Dostál, 2015). Problem-solving is defined as a method for dealing with problems, seeking information to solve problems and finding solutions of problems through systematic and logical thinking (Ahghar, 2012). The problem-solving process needs some steps that include the ability of students to understand the core of the problem, determine the information to solve problems and evaluate solutions (Burkholder et al., 2021). Problem-solving as a way of obtaining solutions to difficulties is divided into four indicators, which are: (1) understanding the problem, (2) making plans, (3) implementing plans, and (4) reviewing solutions (Polya, 1978). The problem-solving process includes the construction of information to connect the gap between the problem and the solution of the problem (Leak et al., 2017). With problem-solving skills, students are motivated to focus on different problems and analyze these problems more broadly. Therefore, problem-solving skills is able to encourage increased creativity from students (Nozari & Siamian, 2014).
Problems in physics often involve components that add the difficulty of the problem such as mathematical manipulation and procedural frameworks to solve the problem. In the process of solving physics problems, it is necessary to coordinate knowledge that connects concepts, mathematical skills, computing, procedural frameworks and other information (Becker et al., 2020). Based on the data, the low problem-solving skills of students is caused by students only memorize the formula and do not understand the physical meaning of the formula (Januarifin et al., 2018). Efforts and strategies are needed to overcome difficulties in solving problems.
Learning multimedia is a form of technology integration to help the improvement of students' problem-solving skills. The development of technology itself is very helpful to improve the learning environment quality as a media or source of learning (Becker et al., 2020). Learning multimedia is a combination of audio and visual media that helps in the process of delivering learning information in a flexible, fast and real-time (Rohendi et al., 2019). Unfortunately, the availability of applications that present physics material in multimedia content is still difficult to find. Therefore, it is necessary to develop smartphone-based learning multimedia that support online learning to improve the problem-solving skills of students (Arista & Kuswanto, 2018). Augmented reality is an immersive technology that can be used to solve problems related to the visualization of physics concepts (Cheng & Tsai, 2013).
Augmented reality is a virtual reality technology that combines objects in the real world with digital information generated by the software (Akçayır & Akçayır, 2017). The technique of augmented reality is to overlay a real object with visual content to produce a visual appearance that seems real to the object. This augmented reality feature uses hardware like a smartphone with camera access to scan and combines objects with digital content (Kravtsov & Pulinets, 2020) (Klopfer & Squire, 2008).
There are two types of AR systems namely Image-Based AR and Location Based AR. The imagebased type was developed from the marker system while the location-based type was developed from the marker less system (Pence, 2011). The way the augmented reality feature works is by scanning the markers and then the tracking system reconstructed with a coordinate system in the real environment. The scanned real object is then processed by the application to be forwarded to the graphics system at the camera position (Cheng & Tsai, 2013).
Augmented reality features can be added to learning media such as worksheet. Worksheet containing assignments accompanied by work instructions according to the teacher's method. With worksheet, students are allowed to discuss and be active in learning activities (Yennita et al., 2018). Worksheet help students to be able to find physics concepts and develop scientific attitudes through discussion activities with other students (Rahmi et al., 2018). Worksheet are intended for students as effective media because they can support students to be more active in the learning process more than just using books (Baihaqi et al., 2021). Based on research using worksheet-based Problem-Based Learning, it is known that worksheets can improve students' cognitive and psychomotor learning outcomes (Yulianti, 2017). However, worksheets are not suitable for use in online learning because students cannot conduct remote experiments. So, it is suggested for new research to combine worksheet with other media to facilitate students in learning process (Kahar et al., 2018). This research will provide novelty about worksheets that will be collaborated with augmented reality features related to the physics concepts of momentum and impulse. Therefore, this study will be developed a learning media in the form of an augmented reality that integrated to the eworksheet to improve students' problem-solving skills.

Research Procedure
The type of research used was research and development (R&D) with a 4D model which included define, design, develop and disseminate steps. This development research was focused on developing e-worksheet media that was integrated with 2D AR features used the Artivive application to improve the problem-solving skills of high school students. The procedure of 4D research shown in Figure 1. The problem-solving skills indicators measured in this study included: (1) understanding the problem, (2) making a plan, (3) implementing the plan, and (4) reviewing solutions. The instruments were then validated using a validation sheet by considering didactic, content, structure and linguistic aspects. This development research was involved respondents of senior high school students who studied with an online learning system. The respondents consisted of 24 students of as the field trial class and 31 students of class X as the operational field class. The subject sampling technique was carried out using a purposive sampling technique.
In the develop steps, first, the AR Physics E-Worksheet media was conducted in the field trial class. Constraints on the media faced in the field trial class were then revised before the media was applied to the operational field test class. The measurement of the effectiveness of using AR Physics E-Worksheet media was reviewed on the increase in the pretest score to the posttest score after using AR Physics E-Worksheet media.

Data Analysis Techniques
The learning instruments used in this research had been validated by physics teachers as the practitioner validator and the expert validator. The results of the instrument validation scores were analyzed using average value. The average score obtained previously was then converted into fourscale interval criteria as shown in Table 1. The problem-solving skills data score were analyzed using the Normalized Gain test to determine the magnitude of the increase in problem-solving skills after the implementation of the AR Physics E-Worksheet (Hake, 1999). The results of the Normalized Gain analysis were then calculated into criteria using Table 2.  Hake (1999) The problem-solving skills data score also were analyzed using statistical descriptive analysis and Paired T-Test with analysis software. Paired T-Test is one method of testing the hypothesis by compare mean differences where the data is not independent (pairs). Before the Paired T-Test is carried out, paired samples must pass the prerequisites for the normality test. The paired values must be assumed to be normally distributed (Hsu & Lachenbruch, 2014). The Paired T-Test analysis was used to determine the effect of using AR Physics E-Worksheet media on students' problemsolving skills. The null hypothesis of this research stated that there is no effect of using AR Physics E-Worksheet media in improving students' problem-solving skills. Before the Paired T-Test was carried out, the data were analyzed using the Shapiro-Wilk normality test to determine whether the data were normally distributed or not.

Results and Discussion
The AR Physics E-worksheet media was prepared with momentum and impulse material for class X at first semester. The AR Physics E-worksheet media presented discussion sheets and practice questions equipped with images that had been programmed using the augmented reality feature. In general, worksheet was used as a media to support experiment activities in offline learning. However, that kind of worksheet cannot support online learning because students cannot do a practicum in the laboratory. The AR Physics E-worksheet provided innovation in physics learning media because it provided a novelty in the form of augmented reality features on the e-worksheet that can support online learning. With the AR feature, students can observe the practical simulation by simply scanning the image on the e-worksheet and then analyzing the results of the practicum simulation. The view of the AR Physics E-worksheet was presented in the Figure 2.
The material on the e-worksheet was adjusted to the curriculum and basic competencies of momentum and impulse material. The content in the e-worksheet contained statements of everyday problems related to momentum and impulses presented by AR features that encourage students to solve the problems. In addition to the E-worksheet, other learning tools such as lesson plans and instruments pretest-posttest questions were also prepared. The lesson plan was structured with the Momentum: Physics Education Journal, 6 (2), 2022, 154 syntax of the guided discovery learning model for online learning. The instrument test was developed based on the indicators of problem-solving skills and related with momentum and impulse problems in everyday life.

Product Eligibility
The instruments were validated by physics teachers as the practitioner validator and expert to assess the validity of the instruments. The validation scores of the instruments were analyzed using the average value and then converted into intervals of validation criteria. The assessment of AR Physics E-Worksheet learning media was assessed based on aspect of content, media construction, media use and media appearance. The corrections given by the validator to the media were then revised before the AR Physics E-Worksheet media was implemented to a trial class and operational class. The validation results by 3 validators of the AR Physics E-Worksheet media were displayed in Table 3. Based on the Table 3, the result showed that the average validation for the AR Physics E-Worksheet learning media obtained a score of 3.73. The score results were then converted into assessment criteria using four-scale interval. Based on the interval, it showed that the result score included in very good criteria so that the AR Physics E-Worksheet media was valid to be used in the learning process of momentum and impulse material.
The lesson plan instrument was arranged according to core competencies and basic competencies for grade X middle school students. The lesson plan lesson plan made for online learning using google meet platform. The validation scores then averaged and converted into intervals of validation criteria. The validation results of the lesson plan were displayed in Table 4.  Table 4 showed that the average score of validation result of lesson plan obtained 3.70. The score is then converted into the validation criteria and is included in the very good criteria. Based on the scores and criteria interval from the validation results, it is known that the lesson plan instrument is valid to be used in the learning process for momentum and impulse material using guided discovery learning model.
The pretest-posttest question items to measure the problem-solving skills of students were arranged based on indicators of problem-solving skills and indicators of achievement of competence for momentum and impulse materials. The pretest-posttest question consisted of 8 problem solving question related to momentum and impulse material in everyday life. The validation results of the question items were displayed in Table 5. The average validation results in Table 5, showed that all question items score were in interval of 3.25 < ≤ 4.00. It showed that each question item was included in the very good criteria and valid to be used to assess students' problem-solving skills for each indicator related to momentum and impulse material.

Implementation of AR Physics E-Worksheet
Learning physics cannot be separated from experimental activities. In contrast to other learning, the learning media used in physics lessons must be able to facilitate these aspects of phy-sics. This AR e-worksheet contains pictures of momentum and impulse phenomena, for example a moving car and colliding objects. The images were programmed with the augmented reality feature. This eworksheet also facilitates experimental activities that cannot be carried out due to online learning. AR Physics E-worksheet provides experimental simulation of the coefficient of restitution. So that students can still observe the simulation and then analyze the data obtained through the augmented reality simulation results from the falling ball.
The images contained in the AR Physics E-Worksheet are images that have been programmed using Artive to be combined with the material explanation video and also the restitution coefficient practicum simulation video. The AR feature displayed phenomena related to momentum and impulse in everyday life and presented a falling ball simulation for calculating the coefficient of restitution. This feature would be active when the image was scanned using the Artivive application installed on the smartphone as shown in Table 6.

E-Worksheet
Augmented Reality Displays The image on the e-worksheet shows a moving car and baseball player. If the image is scanned, an AR video of the problem statement and concept explanation will appear and encourages students to formulate the momentum and impulse formula: and The e-worksheet also contain the image of falling balls for the coefficient of restitution experiment activity. If the image is scanned, a falling ball experiment simulation will appear and students can analyse the data obtained from the simulation.
The pretest and posttest questions consisted of a representation of the problem-solving indicators which had been avowed valid before. The AR Physics E-Worksheet media was distributed to students through the WhatsApp group so that students can still access the AR Physics E-Worksheet outside of learning hours. The augmented reality feature can be accessed using the Artivive application which can be downloaded on the Play store/Appstore via smartphone.
Problem-solving score data was obtained through pretest and posttest before and after using the AR Physics E-Worksheet. The learning process using an AR Physics E-Worksheet for Momentum and Impulse material was carried out for three meetings. The AR Physics E-Worksheet was applied to the field trial class first to conduct trials and revise the media if there are obstacles during learning.
The research was then continued in the field operational class. Before being applied to the test of field operational class, the obstacles encountered previously in the field trial class were fixed. The improvements were adding the manual how to use the Artivive application to scan AR Physics E-Worksheet, adding an example steps of how to solve the problem and also preparing groups for student discussions. From the operational test class, the problem-solving value data obtained through the pretest and posttest then analyzed using Normalized Gain. The average of pretest score before the implementation of AR Physics E-Worksheet was 30.93 and the average pretest score after the implementation of AR Physics E-Worksheet was 84.47. From the improvement of pretest-posttest score obtained normalized gain value of 0.77 which was included in high category of improvement.
The data from each respondent then analyzed using Paired Sample T-Test to determine the effect of implementation of AR Physics E-Worksheet to improve problem-solving skills. Previously, a normality test was conducted to determine whether the data were normally distributed. The results of the normality test were shown in Table 7. The analysis of the Shapiro-Wilk normality test showed that the value of Sig. is 0.073 for pretest score and 0.237 for posttest score. The value of Sig. obtained > 0.05 showed that the data are normally distributed. After the data requirements are normally distributed, then the Paired Sample T-Test was carried out using analysis software. The results of the Paired Sample T-Test were shown in Table 8. Based on the result of the Paired Sample T Test (Table 8), the value of Sig. is 0.000. The value of Sig. obtained < 0.05 then the null hypothesis is rejected. So that there was an effect of using AR Physics E-Worksheet media in improving students' problem-solving skills.
Analysis was also carried out for each indicator of problem solving skills. The increasing score for each problem-solving indicator was shown in Table 9. Table 9 showed that the highest increase occurred in the indicator understanding problem, which was 0.94 in the high category. This was because the AR Physics E-Worksheet provided a visualization of the material so that students can more easily understand the material. For indicators of planning solutions, obtained a value of 0.77 and reviewing solutions obtained a value of 0.87. Both indicators were included in the category of high improvement after we added the problem example and steps to solve the problem. As for the Momentum: Physics Education Journal, 6 (2), 2022, 158 indicators of implementing the plan, the gain value was 0.62 with a moderate increase category. The comparison of the average pretest and posttest scores for the operational field class can be seen in Figure 3.   Figure 3 showed the comparison graph of the improvement in pretest-posttest scores for each indicator of problem solving skills. The AR Physics E-Worksheet media that was developed can help improve the problem-solving skills of students for momentum and impulse material in terms of the response to improve learning outcomes. The augmented reality feature in the e-worksheet helps provide visualization of the problems contained in the e-worksheet so that students got an overview and were able to understand it more easily. Presentation of visualization on the problem helped students to understand the condition of the problem and the variables asked. AR Physics E-Worksheet helps students to understand the concepts and equations, so that it will direct students in determining the plan that will be used to solve the problem. For indicators of implementing the plan, students were given sample questions that practice problem-solving steps ranging from converting the information obtained from the questions into international units to re-substitution until the right solution is obtained and reviewing it according to the concepts of momentum and impulse. The implementing plan indicator has the lowest increase because students are less careful in converting units and less careful in calculations.
The research was relevant with previous research related to the development of problemsolving based worksheets. Worksheets have an effect on improving student learning outcomes and problem-solving skills (Kahar et al., 2018). The results of this study support previous research by (Kravtsov & Pulinets, 2020) related the effectiveness in using augmented reality-based electronic learning media. Meanwhile, the results of the student survey showed the willingness of students to learn to use augmented reality technology. The research by (Wahyuni & Sulisworo, 2020) identified that for physics subjects, the development of technology in learning media is very necessary because physics contains many abstract concepts that need visualization features. This AR Physics E-Worksheet development research also related to the development of worksheets with sensor-based applications on smartphones to assist and facilitate students in understanding physics learning, especially in experimental activities such as detecting physical quantities through smartphone applications. This AR Physics E-Worksheet media facilitates experimental activities that cannot be done during online learning. The learning media that can provide the experimental activity using AR can improve the learning outcomes (Ibanez et al., 2015). The AR feature from the e-worksheet consisted of video explanations of momentum and impulse material, descriptions of phenomena that occurred in everyday life and practical simulations of falling ball reflections. Especially in online learning, the integrated AR e-worksheet media helps in presenting simulation and practical content that cannot be done directly through online learning.
The limitation of this study is that students were expected to have two devices. One device was used to display the AR Physics E-Worksheet and the other device must install the Artivive application to be used to scan the AR Physics E-Worksheet image. The solution to this limitation was that if students only have one scanning device is by displaying the e-worksheet through google meet.

Conclusion
The result of the development research showed that AR Physics E-Worksheet was avowed valid by practitioner and expert validator with validation score 3.73 in very good criteria. The AR Physics E-Worksheet equipped with learning objectives, the manual how to use it, images that can be scanned and display augmented reality features in the form of learning videos and practical simulations, and practice questions that train the problem-solving skills of students. From the result of average score of pretest-posttest, the average value of the students has increased significantly. This is supported by the results of a gain analysis of 0.77 on the high criteria and from each indicator of the increased problem-solving ability. Based on these results of Paired Sample T Test, it can be concluded that there was an effect of using AR Physics E-Worksheet in improving students' problemsolving skills. The highest increase in problem-solving abilities occurs in the indicator of understanding the problem since the AR feature provides visualization related to the problem. Understanding the problem guides students in planning solutions to problems. The indicators for implementing plans obtained the lowest increase, due to the lack of accuracy of students in converting units and calculations. The reviewing solution indicator also produces a high increase because students review the solution with the material explained through AR in the worksheet.