Students’ acceptance of mobile -based assessment

: The effective development of a Mobile-Based Assessment (MBA) depends on students’ acceptance. The aim of this paper was to examine the determinant factor of students’ behavioral intention to use mobile -based assessment. Data were collected from 105 second grade students of a vocational high school through an online survey questionnaire. Partial Least Squares (PLS) was used to test the measurement and the structural model. Results showed Perceived Ease of Use as the strongest direct predictor of Behavioral Intention to Use, followed by Perceived Usefulness. Content and Mobile Self-Efficacy only has an indirect effect. These four variables explain 51.3 percent of the variance of Behavioral Intention to Use.


Introduction
Assessment is an important process in education as students' learning is measured through it (Joosten-ten Brinke et al., 2007). As students' frequent use of smartphone and the internet is increasing rapidly, Mobile-Based Assessment appears to be an alternative to traditional assessment practices. Mobile-Based Assessment (MBA) is an assessment that is delivered through wireless technologies and mobile devices (Nikou & Economides, 2017a). However, effective implementation of any kind of technology depends on user acceptance (Davis, 1989).
There are a lot of educational researchers who tend to work on the impact of Mobile-Based Assessment or the acceptance of it with the purpose to define the variables that might explain the acceptance issue. Mobile-Based Assessment have a positive impact on student motivation (Alioon & Delialioğlu, 2019;Bogdanović et al., 2014;C.-M. Chen & Chen, 2009), learning performance (Dalby & Swan, 2018;Fuad et al., 2018;Kuo-hung et al., 2016;Roschelle et al., 2010) and attitude (Chou et al., 2017;Chu et al., 2010;Wang, 2015).
Users' acceptance of a technology or their behavioral intentions to use technology has been studied and described previously by numerous studies as in the Technology Acceptance Model (TAM) ; the Unified Theory of Acceptance and Use of Technology (UTAUT) (Venkatesh et al., 2003) which is emerged from the Theory of Reasoned Action (Fishbein, 1979) and the Theory of Planned Behavior (Ajzen, 1991); Technological Pedagogical Content Knowledge (TPACK) framework (Rosenberg & Koehler, 2015); the Computer Based Assessment Acceptance Model (CBAAM) (Terzis & Economides, 2011); and Mobile Based Assessment Acceptance Model (MBAAM) (Nikou & Economides, 2017a). Considering the scope of the current research, the MBAAM was adopted and studied with the aim of determining the students' intentions to use a Mobile-Based Assessment in the case of a summative assessment of vocational subject in vocational high school. In this study, the mobile-based assessment used is Questbase which is accessed via mobile phone. QuestBase is a web-based, cross-platform application that provides everything we need in order to create and manage assessments, tests, quizzes and exams, both in on-line and printed modes. Designed as a training and learning tool, QuestBase can also be used for selection tests, psychological tests, satisfaction and opinion surveys, market research and customer feedback (Diseko & Modiba, 2016). QuestBase as an educational tool is able to help teachers to create self-grading quizzes, assign them to students, assess their performance, and analyze results. In addition, as a training tool, it enables organizations to measure knowledge, skills and attitudes securely for certification, regulatory compliance and successful learning outcomes (Fidenia s.r.l, 2020). QuestBase can be accessed from a variety of devices, such as PCs, laptops and smartphones. In this study, all students access QuestBase at school using their respective mobile phones and school Wi-Fi facilities. Figure 1 shows example display of questions on QuestBase and Figure 2 the login page display on QuestBase. In this application, the teacher can provide questions in the form of text or images.  Technology Acceptance Model (TAM) is a model that can measure a person's acceptance of an information technology system. This theory was first introduced by Davis (1989b) who developed the Theory of Reasoned Action (TRA). TRA explains that one's perception and reaction to something, will be able to determine one's attitude and behavior (Ajzen, 1991). Perception and reaction of users of information technology will be able to influence one's attitude in accepting information technology (Ong et al., 2004). Following the logic of TRA, TAM explores the factors that influence behavioral intention to use information or computer systems and explains the causal relationship between two key variables -perceived usefulness and perceived ease of use -and user attitudes, behavioral intentions, and actualization of system adoption and use (Davis, 1985;Scherer et al., 2019). If the technology is considered useful and easy to use, then the possibility of the technology being well received by the target users will be higher. That is, this technology has been well developed and right on target. Perceived ease of use and perceived usefulness are the main factors that can influence user attitudes towards the use of e-learning technology (Pušnik et al., 2011).

Mobile-based Assessment Acceptance Model
Mobile-Based Assessment Acceptance Model (MBAAM) is a development of the Technology Acceptance Model (TAM) in the context of Mobile-Based Assessment. The development of TAM carried out at MBAAM is by adding the construct of facility conditions, social influence, mobile device anxiety, personal innovation, mobile self-efficacy, perceived trust, content, cognitive feedback, user interface and perceived ubiquity value and investigate its impact on behavioral intention / interest to use mobile-based valuations (Nikou & Economides, 2017a).

Research model and hypotheses
This study adopted the Mobile Based Assessment Acceptance Model (MBAAM) and the hypotheses aims to examine the effect of several variables, such as Content (C), Mobile Self-Efficacy (MSE), Perceived Ease of Use (PEOU), Perceived Usefulness (PU) and Behavioral Intention to Use (BIU). Figure 3 shows the research model.

Content
Percieved Usefulness course and assessment questions is optimally challenging, reasonable, appropriate and easily understood (Nikou & Economides, 2017b). We hypothesize that: H1. Content (C) has a positive effect on Perceived Usefulness (PU).

Mobile Self-Efficacy (MSE)
Mobile Self-Efficacy (MSE) as an individual's perceptions of his/her ability to use mobile devices to accomplish particular tasks (Nikou & Economides, 2017b). Based on previous relevant study, students' perceive of mobile learning is easier to use and requiring lesser effort for students who have more experience with mobile technology (K. Chen et al., 2011). Therefore, we hypothesize that: H2. Mobile Self-Efficacy (MSE) has a positive effect on Perceived Ease of Use (PEOU).

Perceived Usefulness (PU)
Perceived Usefulness (PU) is determined as the degree to which a person believes that using a particular system will enhance his/ her job performance (Davis, 1989). We hypothesize that: H3. Perceived Usefulness (PU) has a positive effect on Behavioral Intention to Use (BIU).

Perceived Ease of Use (PEOU)
Perceived Ease of Use (PEOU) is the degree to which a person believes that using a particular system would be free of effort, which is the second major variable of the original TAM (Davis, 1989). We hypothesize that: H4. Perceived Ease of Use (PEOU) has a positive effect on Perceived Usefulness (PU) H5. Perceived Ease of Use (PEOU) has a positive effect on Behavioral Intention to Use (BIU).

Behavioral Intention to Use (BIU).
According to the Theory of Planned Behavior, an individual's behavior can be explained by his or her behavioral intention, which is jointly influenced by attitude (an individual's positive or negative evaluation of the performance effect of a particular behavior), subjective norms (an individual's perceptions of other people's opinions on whether or not he or she should perform a particular behavior) and perceived behavioral control (an individual's perceptions of the presence or absence of the requisite resources or opportunities necessary for performing a behavior) (Ajzen & Madden, 1986).

Participants and Procedure
Participants in this study were 105 students from second grade of public vocational high school. There were 86 males (81.9%) and 19 females (18.1%). The average age of the students was 16.86. All students had already used mobile device for final semester exam (summative assessment) in all subject, include vocational subject. After taking the final semester exam using a mobile device, students were asked to fill out a questionnaire regarding their perception of mobile-based assessment. Their participation was voluntarily and all the data were collected anonymously.

Measures
To examine the research model, we adapted items based on previous studies (Nikou & Economides, 2017b). A modification of the items was necessary in order to be relevant to our study. Moreover, the questionnaire was developed in English and then translated into Indonesian. The data were collected quantitatively through an online questionnaire uploaded on Google Form. The questionnaire included 15 five-point Likert-type scale with 1 = strongly disagree to 5 = strongly agree. These items have been used extensively in several previous studies of acceptance. For Perceived Usefulness (PU), Perceived Ease of Use (PEOU) and Behavioral Intention of Use (BIU) each have three items were adopted from Davis (Davis, 1989). For Content (C) three items were adopted from Terzis and Economides (2011) and for Mobile Self-Efficacy (MSE) three items were adopted from Compeau and Higgins (1995). For this study, statistical analysis and hypotheses were tested using Partial Least Squares (PLS) approach. To carry out the analysis, SmartPLS 3 (Ringle et al., 2015) software was used. PLS was preferred techniques since our sample is small and PLS requirements are more tolerant. Specifically, there are two rules for the sample size: (a) 10 times larger than the number of items for the most complex construct; (b) ten times the largest number of independent variables impact a dependent variable (Chin, 2014). Regarding the first rule, all construct of this study has 3 items. Thus, the minimum recommended value regarding sample size is 30, which is surpassed from our sample (105).
Measurement of reliability and validity are based on the following criteria: (a) Items' factor loadings on the corresponded constructs have to be higher than 0.7; (b) Average Variance Extracted (AVE) have to be higher than 0.5 and the AVE's squared root of each variable has to be higher than its correlations with the other constructs; (c) Cronbach's alpha and composite reliability have to be greater than 0.7 (Barclay et al., 1995;Chin, 2014;Fornell & Larcker, 1981). Structural model and hypotheses are supported by: (a) the variance measured (R2) by the antecedent constructs; (b) the value and the significance (t-values) of path coefficients and total effects (Hair Jr et al., 2016). Bootstrapping procedure is applied to measure t-values. Table 1 shows the item's factor loadings, the AVE, the Cronbach's alpha and the composite reliability. Table 2 presents the correlations among the constructs and the square root of AVE for each construct. All constructs have AVE higher than 0.5 and only one variable has factor loading below 0.7 (i.e. MSE2 and MSE3, but MSE2 deleted from the construct). Cronbach's alpha measures the lower limit of the reliability value of a construct, while the composite reliability measures the actual value of reliability of a construct. Composite reliability is considered better in estimating the consistency of a construct (Salisbury et al., 2002). There are two constructs has Cronbach's alpha below 0.7 but the composite reliability for all construct is higher than 0.7. Results show support the validity and the reliability of the measurement model.  Tables 3 and 4 present the results regarding the hypotheses, the variance measured and the total effects. Regarding Behavioral Intention to Use, this study confirm a direct positive effect of Perceived Ease of Use and Perceived Usefulness. Content through Perceived Usefulness and Mobile Self-Efficacy Perceived Ease of Use is also a positive indirect determinant of Behavioral Intention to Use. Moreover, the analysis shows that Perceived Usefulness is determined directly by Content and Perceived Ease of Use, and indirectly by Mobile Self-Efficacy. Furthermore, Mobile Self-Efficacy have a direct positive effect on Perceived Ease of Use. Thus, the results from the PLS analysis support all hypotheses except the direct effect of Content on Perceived Usefulness (Hypothesis H1). In addition to path coefficients (t value), structural model analysis includes the variance measured (R2) of dependent variables by the antecedent constructs. R2 values of 0.02 considered as small, 0.13 considered as medium and 0.26 considered as large variance (Terzis et al., 2013).

Results and Discussion Result
The model explains 51.3% of the variance (R2) in Behavioral Intention to Use. The total effects of Content (0.055), Mobile Self-Efficacy (0.295), Perceived Ease of Use (0.647) and Perceived Usefulness (0.308) indicate that these constructs are important determinants of the Behavioral Intention to Use. Furthermore, Content (0.179), Mobile Self-Efficacy (0.267) and Perceived Ease of Use (0.585) explain 50.3% of the variance in Perceived Usefulness. Finally, Mobile Self-Efficacy explain 20.8% of the variance in Perceived Ease of Use. R2 of Behavioral Intention to Use and Perceived Usefulness considered as large variance since it is higher than 0.26 and R2 of Perceived Ease of Use considered as medium variance since it is higher than 0.13 but below 0.26.
The findings of the study are in line with previous research about technology acceptance (Ajzen & Madden, 1986;Compeau & Higgins, 1995;Davis, 1989;Nikou & Economides, 2017a, 2017bTerzis & Economides, 2011). This study confirms that Perceived Ease of Use and Perceived Usefulness affect students' intention to use Mobile Based Assessment. Moreover, the study examines and confirm the impact of Mobile Self-Efficacy and Content but the construct of Content found not significantly supported the intention to use mobile devices in assessment.

Discussion
Considering the important role of students' adoption of various technologies for their implementation and sustainability, the current study attempted to investigate vocational high school students' acceptance and usage of a mobile-based assessment by using MBAAM as the theoretical framework. These findings indicated that the significant predictors of students' behavioral intentions to use mobile-based assessment in order of relevance are Perceived Ease of Use and Perceived Usefulness. Thus, intention to use mobile-based assessment depends on the students' fulfilment of its ease of use and value obtained by its usage. On the other hand, the findings also reported that Mobile Self-Efficacy as significant predictors of Perceived Ease of Use. Hence, students' Perceived Ease of Use depends on their Mobile Self-Efficacy. Moreover, the construct of Content also found significant predictors of Perceived Usefulness.
H1 was hypothesizing that Content (C) has a positive effect on students' Perceived Usefulness (PU). The results show there is not enough support for these hypotheses (t-value 1.454 < t-table 1.96). It means there is no positive effect from Content to students' Perceived Usefulness. This results is contrary to previous research (Nikou & Economides, 2017b;Terzis et al., 2013) which states that Content has a positive effect on students' Perceived Usefulness. In this study the majority of students thought that questions on the MBA are unclear and not easy to understand, questions on the MBA are not in accordance with what has been taught, also the picture on the MBA questions is unclear and not easy to understand.
H2 was hypothesizing that Mobile Self-Efficacy (MSE) has a positive effect on Perceived Ease of Use (PEOU). The results confirm a strong support for this hypothesis (t-value 5.598 t-table 1.96). There is 58.66 percent of students felt they were proficient in using smartphones before taking the mobile-based assessment. This results is in line with previous research which states that students do not feel any difficulties in mobile-based learning if they already have experience with mobile devices (K. Chen et al., 2011).
Similarly, H3 was hypothesizing that Perceived Usefulness (PU) has a positive effect on Behavioral Intention to Use (BIU). The results show support for these hypothesis as well (t-value 2.699 t-table 1.96). This means that if someone considers a technology that is useful, then he will be interested in using the technology again. This is in line with previous research (Landry et al., 2006;Teo, 2009) Moreover, a significant effect of students' Perceived Ease of Use (PEOU) on their Perceived Usefulness (PU) was also observed (H4: t-value 4.813 t-table 1.96), showing a strong support for H4. A positive effect of Perceived Ease of Use on the Perceived Usefulness of a mobile-based assessment was found and this is in line with previous research (Chin & Todd, 1995) Finally, H5 was hypothesizing that students' Perceived Ease of Use (PEOU) has a positive direct effect on their Behavioral Intention to Use (BIU). The results show a strong support for this hypothesis (t-value 4.318 t-table 1.96). This means that if someone considers a technology that is easy to use, then he will be interested in using the technology again. This is in line with previous research (Padilla-Meléndez et al., 2008;Van Raaij & Schepers, 2008). In this study only 6.66% of students were not interested in using mobile-based assessment technology anymore.

Conclusion
This paper aims at examining the determinants factor of students' behavioral intention to use mobile based assessment. The results showed Perceived Ease of Use as the strongest direct predictor of Behavioral Intention to Use, followed by Perceived Usefulness. This means that if the system were easy to use, students would be more likely to continue to use it. Therefore, mobile-based assessment systems have to satisfy student's ease of use every time that they interact. This parameter can be achieved with systems that are frequently updated with easier user interface. However, this condition is fading by the time the student learns to use the system and they does not find important anymore that the system is easy to use. In other words, parameter of ease of use affects users'