CrossQuestion Game : A Design of a Group-Based Assessment Tool to Enhance Student Motivation during Pandemic

This paper explores the effect of applying gamification and flipped classroom approaches through our group-based assessment game, the CrossQuestion, in the course of IT in Business. The course teaches basic IT fundamentals and their application in different functional areas of business and management. In Spring 2020-21, we delivered this course through Moodle platform, integrated with the Zoom video communication tool, to introduce the CrossQuestion game as supplemental resources to engage students. We conducted measurements using the Instructional Materials Motivation Survey scales to verify the game's learning effect. We divided students into an experimental group (85 students who played the CrossQuestion game through gamified flipped classroom session–Spring 2020-21) and a control group (60 students who previously underwent lecture-based instructions and individualized formal assessments–Spring 2019-20). The analysis of students’ grades confirms improvement by applying gamified flipped classroom group-based assessments in the learning process. The students’ questionnaire also confirms that group-based assessments can improve students’ motivation. We developed a game system that was attractive to the students, implying that it can be an effective instructional and recreational material to boost morale, increase collaboration, enhance engagement and socialization opportunities, especially during this challenging pandemic.


I. INTRODUCTION
Starting January 2020, students, academics, and educational institutions worldwide faced aberrant phycological, technological, and physical challenges because of the COVID-19 pandemic. The UAE had its first COVID-19 lockdown in March 2020, and universities underwent drastic measures to comprehend associated challenges. With their well-established programs and delivery models, these universities became less effective in sustaining the motivation and engagement of the students' learning process. It became even a more significant challenge for us to teach IT fundamentals to students with a non-IT background in a completely online environment. Students in these circumstances are often overburdened with too much Manuscript  technical information on topics not directly relevant to their field of study.
Applying the right combination of approaches such as gamification and flipped classrooms might help make the learning process more enjoyable, meaningful, and engaging to students and improve the outcomes of their studies [1], [2]. We have applied gamification, flipped classroom approaches, and explored their effects through our group-based assessment web application, the -CrossQuestion‖. An undergraduate cohort from multiple colleges and programs enrolled for the IT in Business course in the College of Engineering & IT Ajman University. The course explored various topics, including introducing IS and IT, e-commerce, telecommunication infrastructure, e-business, security and ethical issues, and global management through IT. The key aim of the course was to explore critical areas of IT at a high level and highlight its relevance to business. This paper will present the process we used to perform the flipped classroom gamification through the CrossQuestion game. It will specify the design of our gamified flipped classroom and the main gamification components of the CrossQuestion game associated with the course. Although we used the CrossQuestion game explicitly in the IT in Business course, it can be applied to any class with little or no modifications. We used the point-scoring gamification element to reward or penalize groups based on their actions and utilized leaderboards to display groups' positions while in the competition. The two identified hypotheses for this study: gamification can positively enhance student engagement and motivation. As a result, it can improve students' grades. We have performed this experiment in the spring semesters 2020-21.
We organized this paper as follows. The second section reviews the related work in gamification, flipped classrooms, group-based assessments, and its application in education. The third section of this paper describes the proposed gamified flipped classroom game, the CrossQuestion, and its implementation. In the fourth section, we will provide the research methodology explaining the application of CrossQuestion through a combination of the gamified flipped classroom experience for the IT in Business course. We then offer findings, discuss them in the fifth section, and conclude in section six with limitations and future studies.

II. LITERATURE REVIEW
Games have played an essential role in education. Researchers have shown the importance of game-based learning and its positive effect on education [3]. A study [4] presented that using a game increases learning effectiveness and provides strong motivation. Their finding implies that learners can achieve a high level of satisfaction during the game process. In addition, the game system model shows a high level of attractiveness and improves learners' engagement.
In most recent times, flipped classroom model has emerged as a teaching method to encourage student engagement and a more active learning model in higher education [1]. The flipped model is pre-class self-learning, which refers to delivering lecture resources to students before class. Then, during a session, the teacher performs group discussions, exercises, or projects [5]. Many studies [6]- [8] have stated the effectiveness of flipped learning in improving student achievement and academic performance compared to traditional methods. Therefore, researchers have investigated innovative strategies and technologies to make flipped learning more efficacious [9]. Integrating gamification strategy to the flipped classroom can be a practical learning approach [10], [11] on the students' learning achievement and motivation.
Group assessments have become an increasingly popular approach for testing students' understanding of subjects [12]. Many universities encourage their academics to undertake more group and peer evaluations to facilitate learning, especially academic knowledge [12]. The result shows that most students gain experience working in a group, facilitating learning, theoretical knowledge, and collaborative abilities [13].
A study [14] explored different factors to the individual's contributions in group work. These factors include gender, age, academic year, and group work preference. Besides, the study also observed group factors such as the duration of group work, group size, and a lack of individual assessment methods. The study showed that the only factor that influenced group work contribution is an individual assessment method.
The importance of incorporating group-based assessments comes from the fact that learning itself is inherently a social activity. Using group-based assessment aims to develop teamwork and communication skills [12]. The results suggest that repeated exposure to cooperative learning benefits social inclusion, peer learning, and transferable skills [15].
Overall, it requires many approaches to ensure the group work's evaluation and reflect on the student's contribution to the group's work [16]. However, selecting the suitable techniques to implement these types of group assessments can be challenging. Hence, the need to train academics to promote a collaborative learning environment encourages the students to work with each other [17]. It is also imperative to incorporate technologies that allow the students to communicate and collaborate more efficiently [18].
The research results [19] emphasized the importance of technology as an essential resource to model implementation and its efficient use by instructors. Research [20] established that group scores are higher than individual test scores using technology. Using a design-based research method, another study [21] implemented a cloud-based architecture based on a mobile application that evaluated the effective collaboration between students. Their findings have shown that the application can promote interaction, collaboration, and motivation among group members, positively impacting using technological tools in group assessment.
ARCS is an instructional model developed by Keller [22] which focuses on motivation. Its application can improve and maintains the students' learning motivation. This model is essential for e-learning since motivating learners in an online course is more complex than face-to-face classes. There are four critical factors in the ARCS model, Attention, Relevance, Confidence, and Satisfaction.  Attention: It refers to the learners' interests. Teaching materials used can attract and maintain learners' attention and interest  Relevance: A successful course design must show the usefulness of the instructional materials so that learners can connect the content with the real world. Learners get more motivated if the course content has a practical application in real life.  Confidence: This factor focuses on success expectations among learners. Thus increasing the success expectation levels among learners will increase confidence and, therefore, their motivation. Thus learning process is affected by the status of the material of difficulty.  Satisfaction: There is a direct relationship between motivation and satisfaction. Satisfied learners can achieve more in their learning process. Based on Keller's ARCS motivation model, we used and customized the Instructional Materials Motivation Survey (IMMS) [23] to determine if our game application can increase student's motivation to learn. The IMMS uses 36 questions answered on 5-point Likert scale to measure the motivational responses of students (1 = Strongly disagree, 2 = Disagree, 3 = Neutral, 4 = Agree, and 5 = Strongly agree).

A. Concept
The concept behind the CrossQuestion application is to encourage a team-based atmosphere where students can work in a group to prepare and compete with other student teams as part of their learning process.

B. Prerequisite and Rules of the Game
For the CrossQuestion to be implemented in a classroom environment, some prerequisites:  An instructor needs to provide sufficient time for the students to read the required material or resources.  Material or resources provided for the preparation should apply to the enrolled students' course/program/field context.  All students should have the same consistent material or resources for their preparation.
In addition, the game follows some ground rules.  Lates submission of questions results in the disqualification of teams to take part in the competition.  An instructor needs to review submitted questions before accepting them for the competition. The instructor provides no notification for the rejected questions, and the opponent teams get awarded based on the surprise stage, as explained in the next section.  Each qualified team starts with 100 points in the competition.  All qualified teams play in sequence. Team 1, then Team 2, then Team 3, …, Team 6.  Each team can only ask one question during their turn.  Each team must have ten questions in their Arsenal -four questions of 5 points, four questions of 10 points, and two questions of 20 points.  In their turn, a challenging team chooses an opponent team to ask a question.

C. The Sequence of the Game
The sequence of stages and GUI interface (Fig. 1) include: 1) Starting Stage: The game starts with 100 points for each team and ten questions. 2) Selection Stage: First challenging team starts its turn and selects the opposing team and the question they want to ask the opponent team. This question can be of 5 points, 10 points, or 20 points. 3) Response Stage. If the opponent team answers the question of the challenging team successfully within 25 seconds, the game will add question points to their total team score while the challenging team loses the same question points from its total score. If the opposing team cannot answer the question, the game will subtract question points from the team's total score and added to the challenging team. Stage 2 and stage 3 are repetitive and will continue until we meet the ending stage conditions. 4) Surprise Stage: If the challenging team picks up a question from their arsenal already rejected by the instructor (for many reasons during the pre-competition review), question points get awarded to the selected opposing team.

5)
Ending Stage: A team with a score of fewer than 50 marks gets eliminated immediately OR, if all teams have asked their questions OR, the lecture time is over. The team with the highest mark is the winning team.

IV. RESEARCH METHODOLOGY
Sixty undergraduate students from Spring 2019-20 semester and eighty-five undergraduate students from Spring 2020-21 semester were randomly selected and enrolled from different educational programs and colleges for a mandatory first-year course of -IT in Business‖ in the College of Engineering and IT Ajman University. The age of the students in both cohorts was between 24 to 29 years. The gender ratio was 56% females in Spring 2019-20 and 54% females in Spring 2020-21. The course explored various IT topics, including introducing IS and IT, e-commerce, telecommunication infrastructure, e-business, security and ethical issues, and global management through IT. The key aim of the course was to explore critical areas of IT at a high level and their relevance to business.
The same instructor delivered the same course content to the Spring 2019-20 cohort (the non-gamified cohort) in a traditional format with two lectures per week and a Q&A time. The assessments included two formal online quizzes, a midterm, an assignment, and a final exam. The Spring 20-21 cohort (the gamified-flipped classroom cohort) delivery format included one lecture day and one activity day per week. The lecture summarized what to cover in the material, and then students had to read the resources coming prepared for an activity day.
On the activity day, students' groups compete through the CrossQuestion application. These gamified activities supplement the primary course assessments, including two formal online quizzes, a midterm, an assignment, and a final exam. The primary purpose of these gamified-flipped classroom activities was to provide additional learning opportunities through games to understand the relation between IT and business. To evaluate if gamified-flipped classroom implementation was successful, we identified two hypotheses.  H1: Gamified-flipped class delivery of the IT in Business course positively influences students' grades.  H2: Gamified-flipped classroom delivery of the IT in Business course positively influences students' motivation.
To test H1, we collected the grades of the enrolled students in the -IT in Business‖ course for Spring 2019-20 and Spring 2020-21 semesters and performed their comparison using an unpaired two-tailed t-test. To compare the students' samples of unequal size, we need to check the effect on the robustness of the equal variance assumption. However, there is no best rule of thumb for how unequal the sample sizes need to be for heterogeneity of variance to be a problem [24]. Such a test is the most suitable when the underline distribution is normal and when sample sizes are large for any distributions [25].
To test H2, we presented the Spring 2020-21 cohort with the IMMS questionnaire using 36 questions BEFORE and AFTER the course delivery. These 36 questions (Table I) in four categories are Attention (ATTEN), Relevance (RELE), Confidence (CONF), and Satisfaction (SAT). We send the first survey to the Spring 2020-21 cohort at the start of the semester and then the second survey after the final exam. These surveys were anonymous, and we received 35 responses from 85 students for the first survey, and the same 35 students responded to the second survey. We received ten additional responses for the second survey but did not consider them because of the absence of the first survey. We performed a paired two-tailed t-test for the comparison purpose. Fig. 2 summarizes the research approach for this study.  The design of formal and gamified assessments looks dry and unappealing

RELE4
The content of this course is relevant to my interests

ATTEN6
The way the information is arranged in this course helped keep my attention

RELE5
There are explanations or examples of how people use the knowledge in this course

CONF6
The activities and assessments in this course were too difficult 20 ATTEN7 This course has things that stimulated my curiosity

SAT3
I really enjoyed studying this course

ATTEN8
The amount of repetition in this course caused me to get bored sometimes

RELE6
The content and style of assessments used in this course convey the impression that its content is worth knowing 24 ATTEN9 I learned some things that were surprising or unexpected

CONF6
After working on this course for a while, I was confident that I would be able to pass all course related assessments

RELE7
This course was not relevant to my needs because I already knew most of it

SAT4
The working of feedback after the assessments, or of other comments in the course, helped me feel rewarded for my effort.

ATTEN10
The variety of classroom tasks, assessments, illustrations, etc., helped keep my attention on the hands-on activities 29 ATTEN11 The style of delivering lectures and conducting the assessments is boring 30 RELE8 I could relate the content of this course to things I have seen, done or thought about in my own life

ATTEN12
There are so many words on each lecture slide that it is irritating

SAT5
It felt good to successfully complete this course

RELE9
The content of this course will be useful to me 34 CONF8 I could not really understand quite a bit of the material in this course

CONF9
The good organization of the content helped me be confident that I would learn this material

SAT6
It was a pleasure to work on such a well-designed course For the H1 hypothesis stating that gamified-flipped class delivery of the IT in Business course positively influences students' grades, we compared the final grades of Spring 2019-20 non-gamified cohort with the final grades of Spring 2020-21 gamified-flipped classroom cohort. We performed a Shapiro-Wilk W test through an SPSS statistical package with a p-value of 0.981 and a 0.923 for non-gamified and gamified flipped classroom cohorts. Based on the normality result, we selected an unpaired two-tailed t-test for H1. We performed Levene's test to check the effect of unequal sample sizes on the robustness of the equal variance assumption. We found no significance.
The result is significant at p < 0.05. Table II also presents this considerable increase in the means of gamified-flipped classroom cohort compared to the non-gamified cohort.
For the H2 hypothesis, we used IMMS scale-based questionnaires. We received 70 responses, including 35 responses out of 85 students for the first survey, and the same 35 students responded to the second survey. We performed an internal consistency check using Cronbach's alpha. For the BEFORE survey with 30 items (α = .90), the AFTER V. FINDINGS AND DISCUSSION survey had 36 items (α = .88). After verifying the internal consistency, we performed the Shapiro-Wilk W test through SPSS with a p-value of 0.556 and 0.294 for BEFORE and AFTER survey data. Based on the normality result, we performed a t-test for H2 (paired two-tailed).

VI. CONCLUSION, LIMITATION AND FUTURE STUDIES
The COVID-19 pandemic has posed challenges both for the students and the academics: emotionally, psychologically, technologically, and physically worldwide. For us, it became an even more significant challenge to teach courses such as IT in Business in a completely online environment to students with no IT background. We proposed combining gamification and flipped classroom approaches to develop a game, -CrossQuestion,‖ to engage students in a competitive group-based environment.
During the Spring 2019-20 and 2020-21 semesters, we experimented with 60 and 85 randomly selected and enrolled students to measure the effects of gamified flipped classroom approaches and group-based CrossQuestion assessments.
Spring 2019-20 cohort lecture were delivered in a traditional format with two lectures per week and a Q&A time. For the Spring 2020-21 cohort a gamified flipped classroom approach was applied through the CrossQuestion game. After providing a quick overview of resources every week, the lecturer assigned the students to read and prepare ten MCQ questions of varying degrees of complexity and points. The lecturer used the CrossQuestion game to present these questions to other student groups during the competition. Other elements of the gamification included leaderboard, award points, penalty points, and badges. Students were also engaged in other hands-on activities per week, not directly linked with the application.
For the H1 hypothesis, we compared the students' final grades of Spring 2019-20 (non-gamified) and Spring 2020-21 (gamified flipped classroom). We confirmed through the unpaired two-tailed t-test that the average final score of students in Spring 2020-21 was 9.577 points higher than Spring 2019-20. For the H2 hypothesis, we used the IMMS scale to measure motivation through attention, relevance, confidence, and satisfaction. Based on the results using paired two-tailed t-test, the average motivation score of the students after the application of gamified flipped classroom course delivery increased by 15.64 points.
In their study, Oe, Takemoto and Ridwan, [26] gathered interview data of 24 students to identify a significant drawback of gamification in an online environment related to students' difficulties taking notes while playing games. As a result, this made them unconfident and uncertain about the learning outcome. Another study [27] attempted to reinforce laboratory topics by engaging students in an interactive labventure environment. However, they could not motivate real engagement among the students, and they exhibited overall lousy performance. However, our result has shown that a combination of the flipped classroom for pre-classroom students' preparation and gamification during the classroom activities can overcome many issues, as highlighted by [26], [27].
A review of gamification applications by [28] presented various cases where researchers either used their pre-existing platform for the gamified experience or developed their gamified applications. As per the review, in most cases, the gamified activity was well-received by learners, considered effective, educational, and engaging, and in some cases also fun [28]. Our study has also presented a case of positive effects of gamified flipped classroom approach using our purpose-built game the CrossQuestion to enhance the students' motivation by building confidence, improving the relevance of the course content and its perceived usage in students' functional areas, overall attentiveness during the classroom session, and intrinsic and extrinsic satisfaction.
This study has only applied the CrossQuestion game for one course, and the result cannot be generalized unless multiple other studies are conducted through the same process as described in this paper. Therefore, future studies should use this game through gamified and flipped classroom approaches in other academic programs such as medical, architecture, professional training.