An Implementation and Evaluation Report of the Active Learning Method EduScrum in Flipped Class

This paper describes an implementation and evaluation report of the active learning method eduScrum in flipped class. The eduScrum test class was organized by volunteer participants of the master program Automotive Software Engineering at Chemnitz University of Technology, Germany. The “AUTOSAR based software design” seminar was selected as course for implementation the eduScrum method. The eduScrum method applied during the winter semester 2018/19 and the summer semester 2019. In the first round 30 of 80 students registered to the eduScrum class, and during the second round 40 of 85 students selected the eduScrum test class. The selected course is usually taught in a flipped class format. That was a challenging condition for the implementation of eduScrum. Due to this constrain the traditional way of implementing the eduScrum method needs to be modified and adapted to the given situation. Therefore, e-learning materials developed and placed into the university Learning Management Systems OPAL. The OPAL is online-based platform for academic teaching and learning at universities of Saxony. To measure the success of the eduScrum implementation the students within the test class did a self-evaluation based on the structure oriented evaluation method. The structure oriented evaluation model focuses on evaluation goals.


I. INTRODUCTION
Nowadays, higher education faces big challenges in teaching. Modern students prefer to follow their individual way of learning and study in university. Unfortunately, the current traditional teaching system and administration environment along with the actual curriculum cannot meet the requirements and expectations of today's students. Therefore, many young people drop out from universities with no degree, usually after the first study year. About 29% of bachelor students and 15% of master students in Germany drop out after the first year of study [1]; nearly 2 million college students who begin college each year will drop out before earning a diploma and only 26% could finish in four years of two years' program in USA [2]. The situation in UK is same: 8.8% of full-time first degree students under 21 Manuscript received March 20, 2020; revised June 14, 2020. This work was supported in part by the DrIVE-MATH (2017-1-PT01-KA203-035866) which is funded by the Erasmus+ Program of the European Union.
U. Tudevdagva is with the Mongolian University of Science and Technology, and with the Chemnitz University of Technology, Straße der Nationen 62, D-09111 Chemnitz, Germany (e-mail: utu@hrz.tu-chemnitz.de).
likely to drop out of university in their first year [3].
The reasons of dropout are complex and differ depending on the countries, universities and individual cases, of course. But researchers listed down main six reasons of dropout: Financial problems, Social or psychological problems, increasing failing courses, De-motivation learning environment, Lack of student support and Study satisfaction, for instance [4]. Professors and universities of European countries started to work to reduce some of the dropout reasons.
The European Union (EU) invests to projects which support students' successful graduation from universities. The Development of Innovative Mathematical Teaching Strategies in European Engineering Degrees (DrIVE-MATH) is an Erasmus+ project where cooperating four universities from Germany, France, Slovakia and Portugal. The main aim of the DrIVE-MATH project is to support and engage students in engineering programs for successful study of mathematics and computer science courses. Chemnitz University of Technology, Germany; University of Lyon, France; Slovak University of Technology, In Bratislava, Slovakia and Instituto Superior de Engenharia do Porto (ISEP), at P.Porto, Portugal are partner universities within the DrIVE-MATH project. These partner universities implementing active learning methods within their classes: eduScrum, Jigsaw, Problem Based Learning (PBL), Hands-On and Think-Pair-Share, are some of the used AL methods. DrIVE-MATH is an example project with target to improve teaching methodologies in higher education and supporting students' success graduation.
The Chemnitz University of Technology implements the following AL methods: EduScrum, Jigsaw, PBL and Hands-On. However, within this paper only an evaluation report of the AL method eduScrum used in a flipped class is presented.

II. THE ACTIVE LEARNING METHOD EDUSCRUM
Actually lecturers are applying lots of different active learning methods in their teaching. The eduScrum active learning method was selected and is implemented in all partner universities [5].
EduScrum is a modified educational version of the project management methodology Scrum. Willy Wijnands as chemistry and physics teacher on the Ashram College is the initiator of this idea. [6]. The main difference of the eduScrum method from the traditional teaching method is to involve students in the teaching and learning process actively and sharing the responsibilities of teaching and learning with the teacher in the classroom. Students playing a key role within the teaching and learning process in this active collaborative education approach.
EduScrum has basic terminologies and some rules for implementation. Some of the terminologies are explained below: • Product Ownerprofessor, teacher or educator who is responsible and supervises the learning subject; • Scrum Mastera student in the EduScrum Team who plays the role of team leader; • EduScrum Teama group of students who uses the eduScrum method during the learning process; • EduScrum Team Membera student of the EduScrum Team; • To-Do Listscheduled plan for the EduScrum Team; • Sprintsubmission deadline for short term results.
The key terminologies and necessary additional background theory about the eduScrum method should be explained during the introduction lectures to the students within the eduScrum class.
The main rule of eduScrum is that students should work as a team which is producing a product with high quality at the end. The product of the eduScrum method is the success in learning and gained knowledge after the course. In traditional way of teaching the teacher is usually "lecturing" the students and most of the time the learning process of students is passive: listening, reading and repeating what the teacher dictates them. In case of eduScrum the students are organized as a team and receiving learning objectives as the "goal" of the learning process. The teacher explains to the students only the main concept and basic theory of the subject, without dictating them what should be done to solve given problems or exercises in class. The students need to work together to solve the tasks and problems and gain knowledge to pass successfully at the end of the subject. The Scrum Master is the main organizer of the team and helps the EduScrum Team Members to work cooperatively and actively together.
The implementation of the AL method eduScrum is modified for the usage in a flipped class. Originally eduScrum is designed for face to face teaching. But for flipped classes face to face teaching is only one part of teaching. The duration of an academic semester at CUT is 15 weeks. Within the first 4 weeks' students attending traditional face to face classroom lectures. During the middle part of the semester the students working six weeks at home alone. In the remaining 5 weeks' students returning to classroom to give a presentation to the peer learners. For using the eduScrum method within the pre-defined course structure it was necessary to apply it to the LMS OPAL. OPAL is used for e-learning within the universities of Saxony, Germany. The Learning Management System OPAL is based on the e-Learning platform OLAT. This is an open source project of the University of Zurich. The Java-based platform was first published in 2004 under the Apache License 2.0. OPAL is used in 11 Saxon universities. The web-based application is already used by more than 60,000 university members of the Free State of Saxony.
As already mentioned within a flipped class students have to work at home for some weeks alone. To support students with 24/7 tutoring services, an online e-learning course in OPAL was created. Fig. 1 shows the example of the OPAL course especially defined for the flipped class situation. All activities of the eduScrum method are specified within the OPAL course as additional elements. In Fig. 2 the structure of the eduScrum elements within the OPAL course are depicted. The eduScrum class starts with an online phase. Hence, the Product Owner (course supervisor) invited the students to register within the EduScrum Team.
The element eduScrum -Group 1 is the cover element for a single eduScrum group and consists of several sub elements. Within the sub element Sprints results of the defined sprints needs to be uploaded while they are ready for discussion. The Presentation and Talk sub element collects presentation files and the preparations of the talks of the EduScrum Members. The Team building sub element is used for the EduScrum Team organization. Within this element students start to introduce themselves and register to the EduScrum Team. After building the EduScrum Team one of the members become the Scrum Master by an open discussion.

III. A METHODOLOGY OF EVALUATION
Main model for the evaluation is the structure oriented evaluation model. This is a new evaluation model which originally focused on evaluation of e-learning [7]-[9]. The structure oriented evaluation model uses logical elements to visualize the defined evaluation goals. Aim of using the structure oriented evaluation model is to measure the overall success of implementation the eduScrum method in a flipped class.
International Journal of Information and Education Technology, Vol. 10, No. 9, September 2020 650 A. The Structure Oriented Evaluation Model The chosen structure oriented evaluation model consists of several steps however very important is the definition of the evaluation goal. Evaluation goal means objective of the evaluation process, which should be reached successfully after the implementation of the applied method.
Therefore, five main objectives also named key goals are defined as followed:  The collected data of the current eduScrum test class are processed according to the pre-defined data processing rule (1) of the structure oriented evaluation model presented in [7]. Advantage of the structured oriented evaluation model is that collected data can be processed by formulas which were developed based on the logical structure of the evaluation. To measure the success of the defined key goals sub goals (can be also criteria) are necessary and visualized by a parallel structure (Fig. 4).

 
The structure oriented evaluation model calculates not only the general evaluation score, by the defined calculation rules with the model also the evaluation score for each key goal and sub goal can be processed.
The general empirical evaluation score is calculated by following formula: ) .
where, ij q is the observed/collected data and ) ( * C Q e is the calculated evaluation score. The theoretical background of the calculation rules and detailed proofs of main data processing formulas, can be find in [7]. The score of evaluation depends mainly from the used questionnaire. Therefore, the questionnaire needs to be well-designed and structured. In general, there is no international standard or theoretical basic guide for the questionnaire design. Hence, questionnaire design is more subjective. In our case main aim of evaluation is to test and quantify the implementation success of the AL method eduScrum in a flipped class. Therefore, it is necessary to design a formal standardized questionnaire. It is obvious to obtain by the questionnaire complete and accurate information as possible.
The first key goal 1 B consists of four sub goals/criteria. Therefore, four questions focusing on the feedback from the students on their opinion and acceptance about eduScrum as a teaching approach.
Second key goal 2 B consists of ten sub goals/criteria. All ten questions are directed to collect students' opinions while comparing the AL method eduScrum with traditional teaching.
Third key goal 3 B consists of eight sub goals/criteria. These sub goals/ criteria focusing on the measurement of the impact of eduScrum to increase the soft skills of the participants.
Fourth key goal 4 B consists of two sub goals/criteria. These two criteria targeted to measure advantages of eduScrum AL method comparing with traditional teaching methods.
Fifth key goal 5 B consists of three sub goals/criteria. Via these three questions feedback from students about future implementation of AL methods in engineering courses is wanted.

B. Data Collection and Processing
For collecting the data an online questionnaire based on the sub goals was developed. The online questionnaire was placed directly to the OPAL course. The announcement for the assessment by an online questionnaire was distributed to the eduScrum test class students via the news function in the OPAL course. In total 40 students registered to the eduScrum class in the summer semester 2019 but only 15 of them responded to questionnaire.
The collected data were processed by the calculation rules of the structure oriented evaluation model.
The calculation result is: Empirical evaluation score .
The score 0.82 indicates that the implementation of the AL method eduScrum was successfully with 82%. These are evaluation scores of key goals. From this data evaluator clearly can see what happened with key goals. All values are bigger than 0 that means goal achieved any how its target. Evaluation scores present how high the performance level of key goal is. Best performance level is the 0.89 score and worst performance is 0.76.
Below the results for the sub goals are presented (see Table  I).  Above listed data show achievement of sub goals. The structure oriented evaluation model calculates all evaluation value from 0 to 1 score and it is very easy to recognize result of evaluation. Moreover, it is understandable to all involved groups in implementation of eduScrum AL method in teaching.

IV. DISCUSSION ON RESULTS
The collected data are not high enough for a quantitative view. Around 23% of the eduScrum students sent a respond to the requested questionnaire assessment. This is one weakness of the actual evaluation. Therefore, the result of this evaluation cannot be used for presenting the opinion of the whole test class.
But, based on the collected data, the evaluation was calculated by the structure oriented evaluation rules for data processing. All result scores for sub goals were above 0.71. It can be summarized that 15 students from the eduScrum class evaluated the implementation of the AL method eduScrum very positive. However, this evaluation cannot be used to interpret the feedback of the remaining part of students but in general it confirms a successful eduScrum implementation at all. Fig. 5 shows that 3 of 15 students evaluated always all criteria with very satisfied. Next 6 students out of the 15 students answered only very satisfied and satisfied. Because of less responds of students these observed data cannot be a final proof for all students' satisfaction. But, these data are fact and shows that for the next round the teacher should spend more attention to attract students to evaluate the implementation process.
This time the evaluation questions were sent to the students after the course during the exam period. So the teacher summarized that can be one reason or less students responds to the evaluation questionnaire. Therefore, for the next round the evaluation questions should be send at least four weeks before the end of the course.
The highest evaluation score of 0.88 was calculated for the question "Active learning is an useful learning strategy" -11 A . This highest score gives the project team positive energy to continue with eduScrum as the AL method in the next round. This fits to expected result of the project investors, too. Therefore, the AL method eduScrum will be implemented in the coming winter semester 2019/20 at CUT.

V. CONCLUSION
The active learning method eduScrum was implemented in a flipped class at the ASE master program of CUT for two semesters. Students of the eduScrum test class gave very positive feedback. One indicator is the increased number of students enrolled to the eduScrum test class in the second semester of implementation. In the winter semester of 2018 totally 80 students selected course the course "AUTOSAR based software design". From registered students 30 students voluntarily decided to change from traditional teaching to the eduScrum test class. In summer semester of 2019 the total number of registered students was 85. Out of this 40 students voluntarily joined to the eduScrum test class. For the second round of eduScrum implementation already two test classes were necessary. This effect shows that students like to test active learning methods in university study.
The evaluation result shows that all five defined key goals of eduScrum implementation reached the target. Therefore, the AL method eduScrum was implemented successfully within the test classes. The highest evaluation score belongs to the first main aim of implementation. It was the question "Students accepted eduScrum active learning method" and the processed evaluation shows 0.89. This result confirms the eduScrum as an active learning method is welcomed by the students and this active learning method should be used in teaching continuously. Therefore, the CUT plans to offer again an eduScrum class in the winter semester of 2020. The learned lessons from the previous test classes helps to improve the implementation of eduScrum in the coming semesters. Most weak points of test classes were: less information about the eduScrum method before the start of the semester at the time of decision which class to choose, students had not enough experience of using the OPAL learning environment