Applying a Formative Assessment Model for a Blended Learning Environment to Promote Students’ Engagement and Motivation

 Abstract —With the advancement of blended learning and learning management systems, online formative assessment plays a critical role in helping students self-evaluate their learning progress and performance. However, there is insufficient research exploring the design of formative assessments in such settings. This quasi-experimental study aimed to assess the effectiveness of an online formative assessment model in a blended learning environment for higher education students. A total of 271 participants were divided into two groups: an experimental group with courses following the proposed assessment model, and a control group receiving traditional model. A survey was conducted at the end of the courses to measure students’ motivation and engagement. Propensity score matching was applied to confirm the sample balance between groups. Based on the matching findings, 78 students in each group were chosen for further analysis. The results from a t-test and qualitative data showed that the proposed assessment model significantly improved students’ motivation and engagement. It highlighted the importance of designing learning and assessment activities in blended learning environment. This model is expected to be applied broadly to test its utility in different majors in higher education.


I. INTRODUCTION
Technology development has created countless opportunities to support education, especially in higher education, with investments in educational applications and learning management systems (LMS) [1].In the post-COVID era, blended learning (BL) has become a trend that takes advantage of LMS resources and face-to-face interaction benefits [2].BL is an integrated system that combines traditional instruction and an online platform to support different instructional methods [3].Based on technology features, BL has demonstrated its effectiveness for postsecondary settings in some aspects, such as unlimited class size, flexible learning space and timetables that can enhance individualized learning [4].However, the definition of BL also impresses the role of instructional design in making the difference between BL and traditional learning [5].For example, Fan et al. [6] proposed that a failure in BL design could cause students' attrition rates, disengagement, and dissatisfaction.Boelens et al. [7] did a systematic review and pointed out four influential factors for designing a high-quality BL course: (1) incorporate flexibility; (2) facilitate interaction; (3) facilitate students' learning processes, and (4) foster an affective learning climate.These factors could be controlled during the course through assessment activities.
Assessment activities are core elements in teaching that allow students to perform their abilities and to receive feedback for learning improvement [8].There are two types of assessment: summative and formative assessment.The latter, which encompasses activities undertaken by teachers and/or students during the process, is considered helpful in providing information for modifying the teaching and learning activities [9].Moreover, formative assessment includes two notions: "assessment for learning" and "assessment as learning."Assessment for learning is related to seeking and interpreting evidence to provide student feedback by teachers and peers, whereas assessment as learning means creating an environment for students to participate in assessment activities by self-assessment [10].In higher education, formative assessment with immediate feedback is particularly important for students who want to make sure of their learning outcomes [11].With the advancement of LMS and blended learning, online formative assessment is considered as a tool to support students in self-evaluating their learning performance, monitoring learning progress, and raising motivation and engagement through a direct feedback system [8,12,13].
Designing successful blended assessment activities requires lectures to consider the suitability of online and offline environments [14].However, a lack of guidelines and references causes many challenges [15].Therefore, based on the concepts of assessment for/as learning, this study designed a series of formative assessment activities in a blended learning environment.A quasi-experimental study was conducted at one university where an experimental group undertook the proposed formative assessment activities, while the control group used traditional ones.After completing the courses, a survey was conducted to assess the effects of students' perception values of the courses' design on their motivation and engagement in the blended learning environment.The independent t test was applied to analyze the difference between the two groups, and qualitative data are also mentioned in the discussion.

A. Formative Assessment in Blended Learning
Blended learning and formative assessment in higher education have attracted great attention recently [4,16,17].The fundamental purpose of formative assessment is to involve students in practices that provide helpful information to improve learning and teaching activities [18].Leahy et al. after working with a number of teachers for one year, proposed five formative assessment strategies that most teachers used in the classrooms [19].They were see Fig. 1: 1) Clarifying and sharing learning intentions and criteria for success.2) Engineering effective classroom discussions, questions, and learning tasks.3) Providing feedback that moves learners forward.4) Activating students as the owners of their own learning.5) Activating students as learning resources for one another.The first strategy will help students and teachers answer the question, "Where is the learner going?".All learning outcomes and assessment criteria for a successful mission should be shared and explained to students before classes.The second and third strategies require teachers to create opportunities for students to show their performances and knowledge through the list of learning activities, such as discussion.The information will then be used to provide feedback for students to improve learning with guidelines for the question "How can we get there?".Besides teachers, peer feedback is also valuable support for learning [20], so the fourth strategy involves the role of classmates in activating learning by finding errors and giving suggestions based on the rubric.The last one is "assessment as learning," which will enhance students' learning by self-assessing their own learning process and outcomes.Using effective peer and self-assessment resources, teachers can save time in finding the information of "where the learner is".However, such assessments require teachers to be fully aware of students' learning intentions and current outcomes.In general, these five strategies cover most aspects of "assessment for and as learning" and are helpful in creating a formative assessment model.
For a blended learning environment, several models are different in time for online or offline phases, such as rotation, flex, self-blend, and enriched-virtual models.The rotation model requires at least one content online, while the others could be provided via an online system [21].The rotation model usually has a fixed schedule of online and face-to-face activities that are suitable for official courses at K-12 and university levels [22].There are four kinds of rotation models: station rotation, lab rotation, flipped classroom, and individual rotation [21].We chose the flipped classroom model to design an online formative assessment model in higher education because it is appropriate to the regulation and students' characteristics in this study [23][24][25].
In a flipped classroom, students enroll in both traditional class and online delivery systems.They could access the learning content and instruction videos via the online system before the discussion in class [21].Based on the five formative assessment strategies and flipped classroom model, we developed an online formative assessment model that was applied in blended learning courses at a university.Because formative assessment is often taken through learning missions, we focus on the activities that teachers and students do in the classroom environment.First, we listed some assessment techniques based on the five strategies and some online assessment techniques proposed by Robles and Braathen [26].The techniques were assigned in either face-to-face or LMS phases.Besides the activities' content, the time allocated for activities between the two phases is vital.Owston and York [27] indicated that in the medium (36% to 40% online) and high (50% online) levels of the blended environment, students have higher positive perceptions of learning and performance than counterparts in other types of learning.A ratio of around 50% is also recommended by other scholars [16].The assessment model designed for this study is shown in Fig. 2.

B. Students' Engagement and Motivation in Blended Learning 1) Engagement
Students' engagement and motivation are essential for learning [28].Engagement is a multidimensional construct containing three elements: cognitive, emotional, and behavioral engagements [29].Kuh [30] further proposed that engagement in higher education should be regarded as an umbrella definition of students' involvement with academically meaningful activities.In this study, we focused on behavioral engagement and followed Kuh's idea to define engagement in blended learning as the level that students actively participated in blended missions [31].

2) Motivation
Motivation is vital to understand students' engagement, especially their learning behavior [32].It is the force that helps students continue to engage, strive, and persist in their International Journal of Information and Education Technology, Vol. 13, No. 11, November 2023 learning paths [33].In a blended learning course, the learning situation and the contextual activities facilitated by the course design are essential for increasing student motivation and engagement [34].Research also indicated that the excellent design of formative assessment could motivate students' learning [17,35].Therefore, in this research, we assessed the effectiveness of the online formative model based on its impact on student's motivation.To sum up, the research question was: do the activities based on the online formative assessment model significantly impact students' motivation and engagement?

A. Setting
The current research was part of a larger study of designing a formative assessment model in a blended learning environment to promote students' learning at one university in Vietnam.The effectiveness of the model was examined via a quasi-experimental study including control and experimental groups.The samples were students enrolling in 5 mixed-grade classes of the educational assessment course.Three of them were randomly assigned to the experimental group and the other two classes were in the control group.The characteristics of participants in the two groups are presented in Table I.In quasi-experimental designs, bias in the treatment effects estimation can be caused by any number of covariates, such as prior academic achievements, gender, etc. [36].In order to reduce such bias, propensity score matching is considered as a proper method to balance participants between groups.In this research, students' gender, grade, major, and GPA were appropriate variables to match the sample.The nearest neighbor matching and caliper algorithm without replacement with a ratio of 1:1 were chosen to calculate using the "MatchIt" package in R.
After matching the data, the standardized mean difference (SMD) varied from 0.04 to 0.19, variance ratios of all factors were around 0.96 to 1.7, and the difference in the empirical cumulative density functions of each covariate between groups was lower than 0.2, indicating a good balance of two groups [37].The distribution of propensity scores before and after matching is presented in Fig. 3. Finally, 78 students in each group were chosen for further analysis.The courses lasted 15 weeks.In control groups, the Moodle system was used to upload missions without interaction activities and feedback.As for the experimental group, the system was designed based on the formative assessment model in Fig. 2. The ratio of time for face-to-face and online LMS was 1:1.The five formative assessment strategies were applied with detailed activities in both face-to-face and online mode, following the model.The experimental courses were designed with clear lesson and assessment plans to confirm the same learning activities in all classes, following the guidelines: 1) Step 1: Clearly define the learning goals of each topic.

B. Data Collection and Analysis
We conducted a survey to gather quantitative information at the end of the course.In addition, qualitative feedback was collected when students completed each topic.After cleaning the data and matching the sample, we employed the R software to analyze the survey data.
First, Cronbach's alpha and exploratory factor analysis (EFA) were conducted to confirm the scale's reliability and validity.
After that, an independent t-test was used to examine the experimental effect with a significant value of .05.Effect size was calculated using the "Practical Meta-Analysis Effect Size Calculator" website based on mean and standard error.

IV. RESULTS
EFA is used to validate the scale using data in both groups.Results of the Kaiser-Mayer-Olkin (KMO) and Bartlett Test (KMO = 0.92, and Bartlett Test p < 0.000) supported the suitability of data for factor analysis.Factor loading of two dimensions explained 70.7% of the total variance.The factor loading values of all variables ranged from 0.65 to 0.85.Cronbach's alpha values for motivation (5 items) and engagement (10 items) factors are 0.928 and 0.94, respectively, which confirm the reliability.

A. t-Test Analysis
The results of descriptive statistics for the motivation and engagement of the groups are shown in Table II.The motivation mean scores for the experimental and control groups are 3.03 (SD = 0.67) and 3.35 (SD = 0.55), respectively.The value for engagement of the experimental group is 3.24 (SD = 0.49), and the value for the control group is lower, 3.05 (SD = 0.56).
Levene's test was applied and the results showed the equality of variances for the two groups in both Motivation and Engagement.As for the results of the independent t-test, the groups' differences are statistically significant (p < 0.05.Regarding motivation, the mean difference between the control and experimental groups is 0.32, with an effect size of 0.56.The value of the effect size for engagement (0.38) is lower.The effect size varies from 0.38 to 0.56, indicating the medium effect of treatment [38].Therefore, applying the formative assessment model can significantly increase students' motivation and engagement.The details with discussion are presented in the next part.

V. DISCUSSION
Although both the control and experimental groups applied blended learning, the difference in designing assessment activities had a significant impact on enhancing students' motivation and engagement.The result is similar to other studies [39,40].It proved the effectiveness of the proposed blended formative assessment model in motivating students to engage in their learning process.
Comparing the design between groups, the most vital differences were interaction in the platform with peer and teachers' feedback, and self-reflection with clear objective learning.For the interaction with feedback, researchers demonstrate that the absence of learner interaction can decrease students' motivation and ultimately lead to dropout in online activities [41].Moreover, online formative feedback is also set for the new direction that engages students in interactive learning environments [42].Some qualitative evidence collected from students' reflections in the experimental group also denoted the role of interaction with peers.For example, students mentioned, "Working with peers and teachers' comments on assignments in Moodle helps me understand what I have to do to revise it, and I would like to fix the product better before submitting again," or "I love the way the teacher created the Moodle forum and Zoom for us to do teamwork together.It actually works.Sometimes, the teacher also joins in and answers our inquiries, which makes our team more efficient."Creating space for students' discussion is also a technique related to strategy 2 in this model.Teachers should combine Moodle forums and discussions virtually to support students' work, and to motivate them to engage in learning activities [43].Fig. 4 provides another example of peer assessment activity in Moodle.The glossary function was used to hold an exhibition of student posters in which students could rate and comment on peers' products.All the posters were presented on one page, and students could choose which they preferred most.These activities greatly attracted students' involvement.As for the other examples of self-reflection with clear objective learning, these activities also enhanced students' motivation and engagement [44].After finishing each topic, students in the experimental group were asked to self-assess their goal, with a scale ranging from 0 (cannot conduct) to 3 (confident to guild your peers to conduct).Fig. 5 shows an International Journal of Information and Education Technology, Vol. 13, No. 11, November 2023 example of reflection.Students were asked to share what they planned to do to enhance their skills and study the following topic with the aforementioned objectives.The feedback function of Moodle was used to collect students' opinions.Students shared, "These are valuable activities that make me understand more about what I have to do in the next step.Moreover, all the reflection process is on Moodle so we can look back to self-assess the improvement through each topic".

VI. CONCLUSION AND LIMITATIONS
This study aimed to contribute to the literature about the role of formative assessment design in promoting students' motivation and engagement in a blended learning environment.The proposed model referring to the five formative assessment strategies and eight guideline steps was expected to be a valuable source for instructors when designing activities in a blended learning environment.The quasi-experimental research results, with effect sizes varying from 0.38 to 0.56, confirmed the effectiveness of applying the formative assessment model in a blended learning environment to enhance students' motivation and engagement.
One limitation of the research design is the non-random sampling.We recognized the potential bias and conducted the propensity score matching method to deal with this issue.After eliminating unmatched students, the final sample confirmed the balance of the two groups in terms of gender, major, GPA, and grade.In future research, it is suggested that researchers could measure students' motivation and engagement during the learning process to understand its change over time and to examine the effect of each strategy on students' learning outcomes.

Fig. 2 .
Fig. 2. The online formative assessment model in the current study.

2 )
Step 2: Design activities to achieve learning goals and choose the corresponding form of organization (LMS/ZOOM/directly in class).3) Step 3: Choose suitable tools, apps, and materials.The Moodle system has many activities, such as exercises, feedback, discussions, quizzes, surveys, and resources (like reading materials and watching videos).4) Step 4: Complete the course's lesson plan by topic or week.The plan is sent out to students at the first session of the courses and adjusted accordingly in the teaching process for each week/topic.5) Step 5: Design assessment tools to gather resources according to each topic/activity.6) Step 6: Set up activities and resources on the LMS learning system.7) Step 7: Implement activities and make adjustments during the process.8) Step 8: Summarize the course, and adjust the plan for the next class.International Journal of Information and Education Technology, Vol.13, No. 11, November 2023

Fig. 4 .
Fig. 4. Example of the exhibition using the glossary function.

Fig. 5 .
Fig. 5. Example of self-assessment and reflection with the Moodle feedback function.

TABLE I :
THE PARTICIPANTS IN THE TWO GROUPS

TABLE II :
MOTIVATION AND ENGAGEMENT RESULT