Developing Gamification to Improve Mobile Learning in Web Design Course during the COVID-19 Pandemic

Thailand is currently facing a widespread third wave of COVID-19 outbreaks in the second quarter of 2021. The government has encouraged social distancing compliance to work from home and study at home policy to mitigate the risks of the pandemic. As a result, educational institutions at all levels must temporarily close their services within the premises. The students need to lean towards the online system independently. Hence, this research aims to develop gamification in the Web Design course to increase the perception and achievement of Information Technology students through mobile learning. The research results showed that the students who registered for the academic year of 2020, the majority’s learning performance improved after adopting the gamification approach via developed mobile application. According to the statistical test (t-Test) results, a significant difference was discovered between pre-test and post-test scores at a significance level (α) of 0.05. The learners also rated the effectiveness of the developed game at the highest level and accepted the developed game with high consensus. In conclusion, the research findings indicate that games can be used as effective instruction media for undergraduate students and online classrooms amidst the situation of the COVID-19 pandemic.


I. INTRODUCTION
Nowadays, the global COVID-19 crisis causing the number of infected people to increase continuously. As of early June 2021, a total of 170,812,850 confirmed cases and 3,557,586 deaths have been reported by World Health Organization (WHO) [1]. While Thailand is currently facing the third wave of the COVID-19 epidemic, with 1,107 deaths out of 165,462 cases and an average increase of nearly three thousand daily cases throughout May 2021 [1]. Such a situation has a wide-ranging impact on all professions, including education, with 145,157 undergraduate students in Thailand for the academic year 2020 [2]. Therefore, the Thai education system imperative transforms into online classrooms towards distance learning models [3], [4]. The purpose is to prevent the spread of the pandemic. Today, some countries or provinces have adopted the lockdown policy to restrict transportation across borders. Many families are affected by economic depressions. Some students who need to stay at home for a long time start to have mental problems, leading to learning loss. A significant reason behind this is the sudden shift from the physical Manuscript received June 6, 2021; revised July 26, 2021. S. Nuanmeesri is with the Department of Information Technology, Faculty of Science and Technology, Suan Sunandha Rajabhat University, Bangkok 10300, Thailand (e-mail: sumitra.nu@ssru.ac.th).
classroom to the online classroom, where students have to study alone and gain less productivity. Studying in the physical classroom can enhance humanistic and communication skills. It teaches students to tolerate, make efforts, help each other, adapt, and observe others. Students are also closely taken care of by the instructors in the physical classroom, unlike online learning, where students learn from pre-loaded materials such as videos on YouTube [5]. Sometimes it is necessary to include engaging content such as animation, which is a popular medium but takes time and cost for creation [6]. In addition, online grades are used in conjunction with online learning, but be aware that students do not take the test themselves [7]. Therefore, the learning process is automatically stimulated by human instinct and the social environment, leading to self-development. However, due to the current circumstances, educational systems must be improved to encourage students to learn more than before.
Nowadays, the smartphone has increased exposure to games among children. It is a powerful tool to encourage students to achieve their learning goals. Games are a significant factor influencing the learning process as they have a positive relationship with the willingness to learn of students as well as creativity and productivity [8]. The diversity of instructional media can also enhance student performance; the more diverse, the better they are [9]. The various patterns of learning may depend on the instructional media. Computer games can effectively stimulate the learning process. So, it is essential to explore how it can be used for improving student motivation and performance [10], [11]. Moreover, motivation enhanced by computer games can develop other cognitive abilities such as memorization and critical thinking [12]. It can be applied to personalized learning which can increase student productivity both in the short run (i.e., higher success rates) and long-run (i.e., an accomplishment of higher education) [13]. Hence, many educational institutions integrate personalized learning [14] during the COVID-19 crisis.
Gamification is the process in which the components of game and game design techniques are used to attract people in non-gaming contexts [15]. It can be said that gamification can transfer the elements and mechanisms of games to non-gaming situations such as classrooms [16]. Games also encourage students to engage in the learning process [17], [18]. Gamification increases students" motivation to explore more content and progress effectively. Games can be used to support dynamic and exciting learning activities, ultimately enhancing student productivity [18]. They are instructional media that can create a student-friendly and relaxing learning environment [15]. Gamification has become popular in the field of education due to its usefulness, such as higher achievement rates and enjoyable learning environment, as well as the rising attention in games [19]- [21]. In language education, games have been used to make the classroom more engaging [22]. Computer-based games are also commonly used in science education; a study shows that computer-based game was conducted on atomic-related subjects. More than 85% of the students supported the use of the game as their instructional media [23].
During the change in learning for a new normal style, learners must stay at home and learn on their own. Most online learnings will still have live, face-to-face teaching and learning through available software such as Google Meet [24], Microsoft Teams [25], and Zoom [26]. There is also content for learners to review or take an online test through the educational institute website, but this is still not enough. Because there are still research gaps that are still missing to stimulate learning in specialties of undergraduate students. However, the course contents could be stimulated learning behaviors by developing courseware in the gamification approach.
Therefore, this research aims to develop a mobile application for learning the Web Design course content using a gamification approach to increase the perception and achievement of Information Technology students in the COVID-19 crisis.

II. RESEARCH METHODOLOGY
This research aims to develop game-based instructional media for the Web Design course for undergraduate students in Information Technology. The research process started from collecting the data on the students" demand for game-based content development. Then, the data was analyzed and applied to the game design and development. After that, the game"s effectiveness and learning outcomes were evaluated.

A. Data Collection
The data collection tools used in this research involved questionnaires and tests. The questionnaires were conducted to explore the potential of the developed game as instructional media for studying at home. Sixty-five Bachelor"s Degree students in Information Technology, Suan Sunandha Rajabhat University, were enrolling in the Web Design course for the academic year 2020. In this research, the documents describing the protocols and research ethics were given to the research participants to sign. Further, thirty-five questionnaires were evaluated the conformity of the content by nine experts in Information Technology and Education Technology. This evaluation is based on the index of the Item-Objective Congruence (IOC) approach, as shown in (1) [27].
where R refers to the score that the rated by experts and N refers to the number of experts. The experts can rate three possible scores: 1, 0, and -1, which were described in Table I. In the event that any of the indicators assessed by the IOC is 0.5 or higher, it means that the question is meet the objective, is consistent with the experts, and can be used for the questionnaire [28]- [31]. In this research, the questionnaire passed the evaluation with an IOC value that was more significant than or equal to 0.8. There is content consistency and is suitable for the gamification approach. 0 There is uncertainty in the content consistency, and it is not suitable for the gamification approach. -1 There is neither content consistency nor appropriateness for the gamification approach.
To collect data from sixty-five students and nine experts, there are five main topics (thirty-five questions) of the Web Design course related to the weekly lesson plans that students wanted to learn based on games. The five topics included 1) colors for web design, 2) webpage layouts and user interface design, 3) responsive web design, 4) coding for web design, and 5) user experience design. The closed-ended questionnaires consisted of 4-point scale questions, in which 1=worst or very inappropriate, 2=bad or inappropriate, 3=good or practical, and 4=excellent or very suitable. The passing score as a criterion for selecting the topics was three on average. The selected topics were analyzed to develop game-based instructional media.

B. Gamification Design
This process began with problem identification and problem-solving based on the gamification approach. In terms of problem identification, it had been figured out that most students in Information Technology could not apply the web design theories to real-life situations. A reason behind this phenomenon was the lack of innovative and easy-to-understand instructional media. Most instructional media were available in texts such as classroom materials and eBooks for computer language learning. Thus, game-based instructional media were proposed for solving this problem. The developed game for the Informational Technology students was available on the Android platform. The visual aids could support the students to learn quickly and comfortably. Above all, the students could immediately recognize their learning outcomes and productivity by checking the correct answers. They did not need to wait until the game under each topic would end.
The game-based instructional media was designed to communicate with visual aids. The learners had to answer the quizzes or tests by dragging and dropping their answers in the blanks. Their answers were immediately checked by the system each by each. Hence, the learners could instantly assess their performance. Visual communication allows the learners to see the contents clearly. More importantly, it would be more challenging to communicate with texts on smartphones. The game-based instructional media was designed based on the analysis of the self-learning approach in the era of distance learning. Therefore, the game-based was designed by applying the progression-based game of structural gamification [32]. Accordingly, the learners will have to learn the course contents from the basics and gradually increase the content level. The gamification-based model was illustrated in Fig. 1. According to Fig. 1, the contents in the course syllabus of the Web Design course consisted of weekly lessons on five topics, their descriptions, and expected learning outcomes. Each learning topic is divided into a different number of subtopics according to the learning objectives and outcomes. Each subtopic contains learning content and a post-study quiz. Furthermore, each learning topic has a test at the end of the lesson. Thus, the learning outcomes can be measured or assessed from the scores obtained by taking the corresponding test according to the learning content of each topic. In addition, the learners can be tracked their learning outcomes from the learning progression at any time. Therefore, these course contents, quizzes, and tests were used to develop the course materials.
The define of the game elements includes points (scores), levels (contents), and leaderboards. The leaderboard is divided into two types: topic and course. The topic leaderboard was the learner ranking divided according to the score of the test results in each topic (one leaderboard per topic). The course leaderboard was the learner ranking according to the total score in the course. The top player (learner) can be demoted or replaced when the last learner scores the same or higher.
For defining the game logic, learners must follow the instruction as follows. First, learners must complete a pre-test before starting to learn any topic content. Second, the learners can study the course content then complete the quiz of each subtopic. Second, once the learners have completed all the subtopic quizzes under any topic, they will take the post-test of that topic to measure their learning outcomes. Learners can take the test only once, and there is no minimum score required to pass the test. If the learners want to retake the test, they must go back to the first step to study each subtopic and complete all the quizzes within that topic. Last, if the learners have completed a test on any topic, the learners can move up to the next higher topic to complete all five topics.
The learners can engage the Web Design course content with the application as follows to play a game. First, the learners study Web Design course content in interactive content through smartphones or tablets. Second, the learners saw the questions with some blanks, and they needed to drag and drop the answers to fill in the blanks. Third, the answers could be immediately checked, then score and update the ranking of the leaderboard. Last, the learners can be view and share their scores on social media such as Facebook.

C. Gamification Development
This work applied the Rapid Application Development (RAD) approach to save cost and development time. Therefore, it is suitable to use in a small development team with the knowledge and ability to the project and quickly develop the work system. The RAD consists of four distinct phases: requirements planning, system design, development, and cutover [33], as shown in Fig. 2. Based on the game design process, the game contents were extracted from the topics in the course syllabus of the Web Design course. The game-based instructional media was created by the Construct 3 [34] software to run on the Android operation system. The homepage became the gate to start the game. Then, there were blanks for dragging and dropping answers. If the answers were correct, the correct marks would appear at the center of the screen. If the answers were incorrect, the incorrect marks would appear, and the answers would bounce back to their original positions. This design will help the learners to avoid getting confused. The displays on the screen were illustrated in Fig. 3 and Fig. 4.   Fig. 3. The interface of the topic is in the color circle and layout. The learners can check their scores and ranking on the leaderboard, as shown in Fig. 5 and Fig. 6. Furthermore, the learners can also share their scores on social media to create pride.

D. The Application Evaluation
The developed application in gamification for the Web Design course was evaluated in three processes: the mobile application evaluation, the game effectiveness evaluation, and the learning outcomes evaluation.
First, nine experts in Information Technology and Education Technology evaluated the developed application by applied black-box testing [35]. It is a technique for software testing that pays attention to the results without the need for the system"s internal mechanisms or composition [36]. For example, the application can perform the tasks or output correctly for the inputs being fed based on their intended purpose. Thus, there are five criteria for mobile application evaluation in black-box testing: functional testing, compatibility testing, usability testing, performance testing, and stress testing. The evaluation result was determined with mean and standard deviation (SD), with an average score between 1 and 5.
Second, the game effectiveness was evaluated by nine experts in information technology and education technology, and all sixty-five students who are enrolled in the Web Design course. The evaluation topics applied the items suggested by Mora et al. [37]. Therefore, there are four criteria for game effectiveness evaluation in this research included: 1) knowledge (must be conformed to the learning objectives, for instance, usefulness), 2) psychology (must be accessible, enjoyable, and engaging for learning), 3) measurement (must be able to measure the learning outcomes), and 4) interaction (must be easy to use and communication). The effectiveness result was determined with mean, SD, and quartiles. The game effectiveness evaluation is based on user satisfaction via an online questionnaire that embeds in the application. They could rate their answers scales from 1 to 5, where 1=strongly disagrees, 2=disagrees, 3=neutral, 4=agrees, and 5=strongly agrees. These answers data were analyzed to examine the mean value and the SD based on the Likert-scale scoring criteria [38], as in Table II. The highest 3.51-4.50 The high 2.51-3.50 The medium 1.51-2.50 The little 1.00-1.50 The least Last, the learning outcomes were evaluated at the implementation phase by sixty-five undergraduate students in Information Technology. During the first week of the semester, these students must complete a pre-test before using the developed game application as their instructional media. Then, at the 14th week of the semester (two weeks before the final examination), there was a post-test for the research samples who had been using the developed game application. Both pre-test and post-test for examining the game-based learning outcomes consisted of fifty questions (ten questions per topic in Web Design course) that shuffled the questions and answers. The data gained from students tested were analyzed using the t-Test on the SPSS software with a significance level (α) of 0.05. If the p-value ≤ α, it means there is a significant relationship between the students learning outcomes of the pre-test and post-test. The hypotheses were set as follows.
H0: The learning outcomes before and after the developed application game-based instructional media are not different.
H1: The learning outcomes before and after the developed application game-based instructional media are different.

A. The Result of the Mobile Application Evaluation
The developed mobile application gamification-based International Journal of Information and Education Technology, Vol. 11, No. 12, December 2021 result was evaluated in black-box by nine experts as follows. First, the "Security testing" had the highest mean at 4.89, with SD at 0.33. Next, the "Usability testing" and "Compatibility testing" had a mean of 4.78 and SD of 0.44. Last, the "Performance testing" and "Stress testing" mean was 4.67, with SD was 0.50. In addition, the mean in total was 4.74 and SD was 0.44, as showed in Fig. 7.

B. The Result of the Game Effectiveness Evaluation
Experts evaluated the game effectiveness among four indicators. The result was found that the "Knowledge," "Psychology," and "Interaction" indicators had the highest mean value at 4.56. Next, the "Measurement" had a mean value was 4.44. All four indicators gave the same SD value was 0.53. The overall game effectiveness by experts, the mean was 4.53, and the SD was 0.51.
For the result of game effectiveness evaluated by users, the "Interaction" indicators had the highest mean at 4.62, together with the SD at 0.49. The remaining indicators, "Knowledge," "Psychology," and "Measurement," had mean values were 4.58, 4.57, and 4.55, with SD values were 0.49, 0.50, and 0.50, respectively. Finally, the users evaluated the overall mean was 4.57, with SD was 0.50. Thus, the Web Design course developed in gamification had the highest effectiveness evaluated by experts and users, as shown in Fig.  8. In addition, the game effectiveness was assessed in quartiles (Q). All interquartile range (IQR) values were not a greater one. Furthermore, the quartile deviation (QD) values were not greater than 0.5 for both experts and users. The result in quartiles showed that the developed game was accepted by experts and users who were using the developed application with a high consensus, as demonstrated in Table  III.

C. The Result of the Learning Outcomes Evaluation
The results showed that their post-test scores were higher than their pre-test scores. The number of the research samples who could answer the questions correctly was compared with their individual scores for analyzing the findings. The findings indicated that the research samples obtained more correct answers after using the developed game-based instructional media, as showed in Table IV. According to Table IV, statistically tested by t-Test, the hypothesis H0 was rejected since the significance value (p-value = 0.0000) was lower than the significance level (α = 0.05). This result showed that the developed game for the Web Design course could affect the learning outcomes of the research samples. After applying the developed application game, the students received an average mean was 43.54, and the SD value was 5.25. This result of the post-test was higher than the pre-test result before applying the developed application game (the average mean of pre-test score was 26.53 with the SD value was 5.41).
In addition, when comparing the achievement of students in the Web Design course from 2018 to 2020, focused on the proportion of those with grade was "F" (Failed) or "W" (Withdrawn). The comparison result was found that the achievement of students who did not apply this developed system resulted in a grade of "F" was 11.59%, and a grade of "W" was 5.80% for the academic year 2018. For the academic year 2019, students achieved the grade of "F" was 14.93%, and grade of "W" was 7.46%. In the academic year 2020, the learning system has entered full online classes with the covid-19 situation. Classroom classrooms have to be closed and bring in game development to help in mobile learning. As a result, the students enrolled in the Web Design course in 2020 got a grade of "F" was 12.31%, and grades of "W" was 6.15%. The comparison of the students who got grades of "F" or "W" for the three academic years was shown in Fig. 9.

IV. DISCUSSION AND CONCLUSION
This research proposed developing the game-based instructional media for the Web Design course for Bachelor"s Degree students in Information Technology. During the current COVID-19 crisis in Thailand, the educational institutions at all levels have been temporarily closed down and shifted to the online classroom system. Therefore, this research designed a visual game as instructional media based on weekly lesson plans and their expected learning outcomes of the Web Design course. The learners may just drag and drop their answers to fill in the blanks, and then the system will immediately check their answers. If the answers are incorrect, the answers will bounce back to their original positions, not allowing the learners to choose them as the answers. The purpose is to avoid the learners getting confused and show them whether they are correct or incorrect right away. The mechanism of the game facilitates and encourages the learning process of the students. Furthermore, they can share their learning experience and score via social media.
The developed mobile application was evaluated in black-box testing. The result showed that all testing had the mean in total was 4.74 with SD was 0.44. It can be said that the developed application has the highest level of application efficiency. Further, the game effectiveness was evaluated based on the four criteria, including knowledge, psychology, measurement, and interaction. The total results showed that the research participants rated the game effectiveness with mean values equal to or greater than 4.50. The SD values are not over 0.51 in all aspects (more than 90%); this means the developed game achieved effectiveness at the highest level. Meanwhile, there was also a consensus among the evaluators. The information technology students accepted the developed game with a high consensus. All values had the IQR no more than one and the QD no more than 0.5. Most importantly, the students" learning outcomes after using the developed game were better than before using the developed game.
Moreover, the result conforms with hypothesis H1 that the learning outcomes before and after students applied the developed game differed at a significance level of 0.05. Besides, the comparison of the percentage of learners who failed (F) or withdrawn (W) in the Web Design course for the academic year 2018 to 2020, it was found that the ratio of students in 2020 was less than in 2019 while same as in 2018. Even in situations where students and teachers must be far away from each other which gradually becomes difficult for teaching and learning for practical courses such as Web Design. The developed learning game helps to make the proportion of learners who failed or withdrawn a few decreased. This might be due to other external factors that could affect the learner"s willingness and ability to learn, which must be researched further.
Hence, the game-based instructional media application was conducted to provide personalized learning to all learners under the online classroom"s conditions due to the current pandemic. Gamification can encourage students to learn in a relaxing and engaging environment actively. The research findings of this study indicate that games had a positive relationship with student achievement. Students tended to have better learning outcomes after using games as instructional media. This work conforms to the studies conducted by Dichev and Dicheva [16], Hamari and Keronen [17], Hamari and Koivisto [18], and Gozcu and Caganaga [19]. The research participants accepted the game as their instructional media. They rated the game effectiveness at a high level with a high consensus. This result conforms to the research carried out by Lutfi and Hidayah [20]. This work also clearly reflects that an online classroom requires instructional media development to support all kinds of learners and situations. As a result, the students may effectively learn by themselves towards online instruction and various electronic media.
For future research, it is essential to investigate the factors affecting the development of game-based instructional media for practice-focused subjects such as programming exercises on smartphones. These factors might help to provide the students with practical skills even under distance-learning conditions.

CONFLICT OF INTEREST
The author declares no conflict of interest.