Investigating the Effect of a Proposed Educational Robot on Students‘ Motivation and Learning of Thermodynamic Concepts

 Abstract —Research in Morocco shows a decrease in motivation and learning of physics subjects for Moroccan middle school students, which negatively impacts their school achievement. The present study aims to identify the effect of using educational robotics on middle school students’ motivation and learning of physics concepts, particularly regarding the concept of temperature. A total of 90 first-year middle school students from a public school in Meknes, Morocco, voluntarily participated in this study. The students were divided into two groups (46 for the experimental, and 44 for the control). The study was carried out during the 2020-2021 school year. The experimental group was taught the temperature lesson using educational robotics (ER). While the control group was taught the same lesson using a conventional teaching method. Two indices were evaluated and analyzed. The index of students’ achievement in acquiring the concept of temperature was measured by pre-test (diagnostic test) and post-test (formative evaluation test). The students’ motivation index was measured by a Likert scale questionnaire. The data collected were analyzed using SPSS software. The post-test results of this study show that students in the experimental group perform better than the control group students. Additionally, the results of the motivation questionnaire show a considerable improvement for the experimental group. The results of the study recommend that teachers integrate educational robotics into their courses to help students achieve better results in the subject of physical sciences.


I. INTRODUCTION
In several countries, research has shown that students perform poorly in science in general and in physical science at all levels of education.
In the international report -Trends in International Mathematics and Science Study‖ [1], which evaluates the science and math abilities of elementary and middle school students, Morocco is ranked at the bottom of the list of countries with the lowest scores in terms of student performance in mathematics and science. Another study conducted by Nasser et al. [2], reveals that Moroccan students generally score poorly in science classes, Manuscript  particularly physics. One more review introduced in (The National Program for the Assessment of Student Achievements) [3], shows that Morocco's science education confronts a lot of difficulties for example the low students' achievement in physical sciences, which has a detrimental effect on the student's school accomplishment. From the analysis of these findings, the teaching of scientific subjects in Morocco requires serious efforts toward improving the school students' success rate. Aiming to ameliorate the quality of education in Morocco, the -National Charter for Education and Training‖ has been adopted since 2002. This charter encourages pedagogical innovations as well as the integration of information and communication technologies (ICT) in primary and secondary schools [4]. The educational robot (ER) is a modern and reliable ICT resource that can be integrated into the learning of scientific concepts. Indeed, ER contributes to the learning of scientific concepts in various disciplines such as mathematics [5,6], Chemistry [7], Programming [8,9], Engineering [10], language learning [11], etc. In addition, ER can also develop skills related to the scientific process that include the evaluation of possible solutions, hypothesis formulation, methodic experimentation, and variable controlling [12]. More research has demonstrated that ER can increase students' interest and engagement in science, technology, engineering, and mathematics (STEM) [13]. Also, the ER is regarded as a powerful ICT tool that can be used to facilitate the active, playful, and constructive participation of students in experimental activities [14]. Thus, ER can motivate the participation of students in various learning activities [15][16][17]. Despite the great importance of the ER in the field of education, its integration into the Moroccan educational system remains almost absent. To overcome this limitation, we first focus on proposing an ER to be used in physics education. Then, we suggest a pedagogical scenario in which the proposed ER is incorporated. Finally, we investigate the influence of using the proposed ER on students' motivation and achievement. It is also important to mention that, to the best of our knowledge, this is the first use of ER in the teaching of concepts related to thermodynamics.
More precisely, we use the proposed ER in the teaching of thermodynamics, with special attention to the concept of temperature. This concept is selected because certain studies show that students have difficulties with the concepts of thermodynamics, particularly -heat‖ and -temperature‖ concepts. Indeed, studies [18,19] illustrate that the term -heat‖ creates learning problems for students. Furthermore, the research [20] notes that most of the students at Al Fateh University confuse the concepts of -heat‖ and -temperature‖, and some of these students believed that there is no difference between -heat‖ and -temperature‖. For these reasons, it seems relevant to create educational environments to ensure a maximum achievement of scientific concepts starting from primary or middle school levels. ICT tools such as numerical simulations can be exploited in the learning of heat and temperature concepts. However, these tools are still limited because the student remains an inactive element during the teaching/learning process. Thus, the student's motivation can be negatively influenced in the learning process. In contrast, the use of ER in teaching activities involves students in the educational process, making them active participants.
The research questions are: 1) ER-based method and Traditional method: which is more effective in teaching the concept of temperature? 2) What is the effect of using ER on the motivation of middle school students to learn physics concepts, particularly the temperature concept?

A. Study Hypothesis
To answer the above research questions, the following hypotheses will be examined. H0 1: There was no statistically significant difference in the performance of middle school students in the experimental group who were taught the temperature concept using educational robotics and those in the control group who were taught the same concept using the conventional method.
H0 2: A physics teaching-learning activity that integrates RE cannot increase students' motivation to learn new scientific concepts.

B. Study Objectives
This study was carried out to achieve the following goals:  Investigate the impact of using the proposed ER on the achievement of middle school students.  Determine the impact of using the ER on students' motivation regarding the concept of temperature.

C. Study Importance
The importance of this study focuses on the following points:  This study showed how this proposed ER can positively affect students' achievement in learning physics.  The proposed ER is beneficial compared to other pedagogical tools because it enhances students' motivation to learn physical science.  This study could be a reference for researchers interested in educational robots as a pedagogical tool.

II. PROPOSED EDUCATIONAL ROBOT
In this section, we present the proposed ER, which will be called -EducThermoBot‖. Specifically, we will use our ER in the learning of the temperature concept by middle school students.
A. The Overall Design of "EducThermoBot" Fig. 1 summarizes the overall design of the -EducThermoBot‖ which is composed of four fundamental units, which are given below.  Studied environment unit: it is the environment that we are interested in observing the variation of its temperature over time.  -EducThermoBot‖ unit: this is the central unit of the proposed ER which contains all hardware and software components.  Communication channel unit: this is the communication channel used to connect the ER and the user. In our case, we use -Bluetooth‖ as this channel will make the robot movement freer.  Robot control unit and temperature display unit: this unit is a graphical user interface (GUI) used to control the robot's movement and displays the measured temperature in real-time. This unit is an Android application we have developed specifically for controlling the proposed ER. Fig. 2 shows the GUI of the developed Android application. When designing the GUI, we have considered its simplicity and ease to understand by students. Fig. 2. GUI of the Android application to control -EducThermoBot‖.
The following subsection presents the fundamental components used in the design of our ER.

B. Basic Components of "EducThermoBot"
In Table I, the main components of the proposed ER -EducThermoBot‖ are summarized with the role of each component.
From this Table I, we can see that the components of -EducThermoBot‖ are very basic, and the total price of our robot is about 12.76 $. Therefore, this robot is affordable and easy to build, and it is developed based on open-source software. These advantages can make the proposed ER accessible for use in public and private schools for hands-on sessions. Moreover, the students themselves under the guidance of the physics or technology teachers can realize this robot. Indeed, the practical realization of the EducThermoBot by the students can be part of a trans-disciplinary project-based pedagogy, especially in the fields of technology (sensors, actuators, motors, etc.), physical sciences (temperature monitoring) and computer science (algorithmic coding). The practical realization of this robot mobilizes many cross-curricular skills acquired by students such as cooperation, communication, learning strategies, creative thinking, and reflexive approach. It is also worth mentioning that our ER can be easily adapted to be used for the measurement of other physics/chemistry quantities such as pressure, gas rate, pH, etc. This can be easily achieved by substituting the temperature sensor with other sensors. Moreover, EducThermoBot is based on a robotics kit with an Arduino board. This technology is easy, accessible, and widely used in general education and technical development courses [21,22]. Furthermore, researchers [23] claimed that the robotic kit with Arduino allowed one to experiment with concrete programming and see the immediate result of a code, and check what happens from the formal specification of a program to its effects on something touchable. The robotics kit thus allows a wide range of educational activities given that it is used both as an object and as a tool, allowing learning -about‖ and -through‖ robotics [24].

C. Prototype for "EducThermoBot"
The -EducThermoBot‖ is practically designed to be used in physics teaching. Indeed, the prototype of -EducThermoBot‖ is presented in Fig. 3. This robot dimensions are 20 cm in length and 15 cm in width and it is equipped with three wheels drive system, which enables it to manipulate the robot in all directions.
To judge the ER and determine the extent of its educational capabilities, and to determine its suitability for students and its suitability for displaying physical content, a questionnaire is developed using the Google Forms tool and then shared online with teachers from Moroccan public middle schools. These teachers integrate ER as part of their teaching process. Moreover, all the teachers who responded to the questionnaire participated in the robotics competition AUROBAT 2021 which was organized by Al Akhawayn University in the city of Ifrane Morocco between March 19 and 20, 2021. The total number of teachers who responded to the questionnaire is 56.
The result of this questionnaire shows that the majority of technology teachers (87.5%) use the ER, on the one hand, to concretize the theoretical background acquired by the students in STEM (Science, Technology, Computer Science, and Mathematics). On the other hand, the ER is employed to stimulate the creativity, imagination, and motivation of students. Moreover, the majority of teacher participants (83.9%) use the robotic kit with Arduino board to build their ER.

A. Design of Educational Scenario Based on the Proposed "EducThermoBot"
The proposed scenario will be equivalent to an existing educational scenario in the Moroccan educational program of the physics subject in the first year of middle school. It is a hands-on physics activity. The objective of this activity is first to determine the temperature of a given object using a thermometer during the heating period. Then, collect the temperature measurements, and finally, construct the curve that corresponds to the collected temperatures over time.
The conventional method used in the Moroccan program to achieve the objective of this activity involves the use of a classical thermometer for measuring the temperature of a body at different instants. Then, collect the recorded temperatures and the timing of each record. Finally, the students trace on graph paper the curve that illustrates the variation of the temperature over time.
In the proposed ER-based educational scenario, students are assigned to orient -EducThermoBot‖ towards the various environments to locate their temperatures (Fig. 4). The orientation of the ER is conducted using a smartphone through the Android application that guides -EducThermoBot‖ toward the studied environments. Finally, students are asked to determine the axes of the graph on their GUI to visualize the real-time variation of the studied environment's temperature.

B. Study Population
Our study population consists of 90 voluntary middle school students, who are distributed according to Table II below.

C. Ethical Considerations
The period in which the research is conducted is an instructional period that follows the logical sequence of first-year middle school students' learning in physical science. Therefore, no compensation is provided to promote students' participation in this physics class as usual. Furthermore, the teachers who participated in this research project are volunteers. However, they can take advantage of the pedagogical material developed by the researcher free of charge. The investigation tool is an individual questionnaire distributed to students in the class after receiving the agreement of the education officials supervising our research and that of the national education authorities.

D. Procedure
To evaluate the influence of using ER on the student's motivation and achievement, two different groups of students are selected: a control group and an experimental one. A comprehensive school management system software called -MASSAR‖ is used to assess the equivalence between both groups. The latter are selected from a public middle school called -Ibn Tofail -in Meknes city in Morocco. The current study is being conducted during the school year 2020-2021. For the control group, the students are required to use the classical activity method to achieve the study goal, and the experimental group is assigned to perform the same activity using the proposed -EducThermoBot‖.
Firstly, to evaluate how employing the educational robot affects students' motivation and learning. We conducted a diagnostic evaluation (pre-test) to assess the initial level of prerequisites of the learners for both groups. Then, the control group (44 students) followed the lesson on temperature according to the pedagogical orientations of the physical science teaching, while the experimental group (46 students) followed the same lesson by integrating the proposed ER. Finally, after the experiment (two hours of hands-on work), the two groups of students are invited to answer the evaluation test and to fill in a questionnaire with a six-point Likert scale, assessing the student's motivation. The assessment test and questionnaire are constructed with two physics teachers. Fig. 5 illustrates an overview of the experimental procedure used in this study.

E. Research Instruments
Three instruments are used in this study. Firstly, a diagnostic test (pretest), secondly, a formative evaluation test (posttest), and thirdly, a motivational questionnaire.
Firstly, a pre-test was developed by the researchers, referring to the physics curriculum. The pre-test was validated by experts (physics teachers with long experience in teaching physical science).
Secondly, the index of the student's performance on the experimental task came from the correction of the posttest developed and validated by experts (Appendix A). This pos-ttest was delivered at the end of the session. The same correction criteria were applied to all the exam copies. Also, all the students' productions are corrected by the same teacher to reduce differences in understanding.
The Cronbach's alpha value is (α = 0.801 > 0.6). Therefore, the internal consistency of the items in our study is strong, and they measure the same skills.
Thirdly to study the impact of ER on students' motivation, we designed a questionnaire (Appendix B) with 8 items on a Likert scale inspired by the questionnaire of researchers [25], assessing students' motivation during collaborative learning. We simply adapted the statements to fit our task, half of them were formulated positively and half negatively. We chose an even scale ranging from 1 (Strongly Disagree) to 6 (Strongly Agree). Given the importance of the motivation questionnaire to this study, we wanted to analyze its reliability using the alpha coefficient (α) developed by Cronbach (1951). This index ranges from 0 to 1 like most other internal consistency indices. Usually, Cronbach's α equal to 0.6 or more is satisfactory; the closer this number is to 1, the higher the reliability of the instrument. The questionnaire was validated directly on the sample (90 students) distributed in both conditions. We then verified, as reported in Table III. The internal consistency of the 8 items measuring motivation (α = 0.937; excellent). Considering that the fidelity index for all 8 situational interest items was high, we will include all 8 items in the analysis of the results, as shown in Table IV.

F. Data Analysis
The learning outcomes of the tests given to the two groups of students, control and experimental, are compared and analyzed in this study using the Student's t-test with SPSS statistical analysis software.
An independent sample t-test is being conducted to examine whether there is a significant difference in performance between students in the control group and those in the experimental group.
A second independent sample t-test is conducted to examine whether there is a significant difference in motivation between the students in the control group and those in the experimental group.

IV. RESULTS
Our principal goal is to investigate how integrating an ER affects students' motivation and learning. To start, we looked at and contrasted the diagnostic test (pre-test) results. Then, we analyzed and compared the data from the evaluation test (post-test). Finally, we analyze and compare the data from the questionnaire on motivation.

A. Results of the Pre-test
To explain the results of this study, we start by comparing the average of results obtained by the students of the two groups (control and experimental). The results obtained are shown in Table V Table V shows that the overall marks mean is about 11.34 with a standard deviation of about 4.5. And the median score is 11.5. According to Fig. 6, The control group's outcomes demonstrate that 20.45% of the pupils achieved marks lower than 09/20 and just 36.36% received positive outcomes. (The mark is higher than 13), an observed average of around 11.13/20. The experimental group showed a similar trend. (19.56%, 28.26%, and 11.54/20 respectively).
Before proceeding with the t-test, we checked for compliance with the 6 basic assumptions associated with this type of parametric analysis. First, our dependent variable, the pre-test, is represented by a score out of 20, where 0 represents a minimum performance and 20 represents a maximum. Second, our dependent variable is composed of two independent groups. Third, we assume independence of observations considering that no student participated in both conditions and that all students are different. Fourth, there are no significant outliers in the distribution. Fifth, the distribution of our dependent variable is normal. Indeed, after testing for normality, the skewness and Kurtosis have non-problematic values, i.e., less than 1, as reported in Table  VI. The Kolmogorov-Smirnov test turns out to be insignificant (p = 0.138), suggesting that the data sufficiently respect normality, as shown in Table VII.  To test the significant difference between the two groups with a normal distribution (Table VII), we used the student t-test. The results are presented in Table VIII. The equality of variances was checked by Levene's test (F = 0.645; Sig = 0.424 >> 0.05) as shown in Table VIII.
The analysis of the results (Table VIII), shows that there is no significant difference between the control group and the experimental group at the 5% level (t(85) = −0.431, p = 0.667 > 0.05). This implies that both groups had the same level of knowledge before the application, which can be explained using the comprehensive school management system -MASSAR‖, whose aim is to ensure equivalence between the groups when forming classes. This result allows us to validate our experimental model based on a pre-test and a post-test.

B. Results of the Post-test
The first specific goal of the study is to determine the effect of using an educational robot on students' understanding of physical concepts.
To accept or reject the hypothesis that educational robotics can lead to better student learning than a lesson without the use of robotics, the scores of the evaluation are statically analyzed using SPSS software. The results obtained are shown in Table IX. The distribution marks of the post-test for the experimental and control groups are shown in the graph given in Fig. 7. The data in Fig. 7 indicate that 59% of the students in the experimental group scored 13 or above. While 52% of the control group students had scores below 9. According to Table IX, the students in the experimental group had a mean score of 13.27 and a standard deviation of 2.657 on the post-test. While students in the control group had a mean of 8.43 and a standard deviation of 4.132.
Before proceeding with the t-test, we had to check that the distribution of our dependent variable (learning) is normalized. In fact, after testing for normality, the skewness and Kurtosis have non-problematic values, i.e., less than 1, as observed in Table X. In this respect, the Kolmogorov-Smirnov test is found to be insignificant (p = 0.196), suggesting that the data sufficiently respect normality, as reported in Table XI.   Following the necessary checks, the independent sample t-test, as reported in Table XII, shows that there is a significant difference between the robotics group (AV = 13.27; SD = 2.657) and the non-robotics group (AV = 8.43; SD = 4.132) (t(88) = 4.489; p = 0.000 < 0.05). Hypothesis 1 is therefore confirmed. Thus, we can say that there is a significant difference in favor of the experimental group in the post-test.

C. Results of the Data from the Motivation Questionnaire
The second specific goal is to compare the impact of using the ER on the student's motivation during a task in physical science compared to the same task without the use of the ER.
To accept or reject hypothesis 2 that the ER can develop the student's motivation. The motivation is measured by a continuous scale, a numerical Likert scale ranging from 1 to 6, where 1 represents a minimum of interest and 6 represents a maximum. The collected data from the questionnaire are analyzed statically with SPSS software and the achieved results are reported in Table XIII.  International Journal of Information and Education Technology, Vol. 13, No. 7, July 2023 The distribution of the means of motivation in the questionnaire for the experiment group and the control one is given in the graph above (Fig. 8).
Table XIII results show that the overall average is 4.04 with a standard deviation of 1.31. The results also revealed that the experimental group's average (4.68) was higher than the general average while the control group's average (3.33) was lower. According to the graph (Fig. 8), much of the average of the experimental group is between 4 and 5, while the average of the control group is between 3 and 4.
Again, before proceeding with the t-test, we checked that the distribution of our dependent variable (motivation) is normal. Indeed, after testing for normality, the skewness and Kurtosis have non-problematic values, i.e., less than 1, as reported in Table XIV. The Kolmogorov-Smirnov test is found to be insignificant (p = 0.23), suggesting that the data sufficiently respect normality, as shown in Table XV.   Following the necessary checks, the independent sample t-test, as reported in Table XVI, shows that there is a significant difference between the robotics group (AV = 4.68; SD = 1.0861) and the non-robotics group (AV = 3.43; SD = 1.2367) (t(90) = 5.367; p < 0.05). Hypothesis 2 is therefore confirmed.

V. DISCUSSION
This study used an educational robot to motivate and improve their achievement in learning thermodynamic concepts. The results were obtained by collecting and analyzing data from the pre-test and post-test evaluations, as well as a questionnaire survey. We can summarize our discussion in the following points: Firstly, this study shows that most of the students in the experimental group (59%) have grades above 13/20. This result is due to the teaching of physical science to the students in a modern way that contains ER that helps the students to understand the physical concepts. Besides, it makes the process of learning physical science, especially the temperature part, easy, enjoyable, and fun. The result of the present study agreed with the study of Barrera [26] which shows that the use of ER promotes active learning through a set of cognitive processes (perception, presentation, imagination, thinking, memory, and speech). In addition, the study [27] shows that ER brings benefits to the student such as teamwork, by involving students in collaborative experiences. In this activity, the student performs a series of tasks such as analysis and logical representation of data to solve problems.
Secondly, the average of the students' motivation for the control group was 3.43, placing this motivation as negative (our Likert scale of the motivation questionnaire had an even number of categories distributed from 1 to 6, where 3.5 would be the median line). The mean of the experimental group was 4.68, thus placing this interest as positive. Furthermore, the difference in the variance of student motivation confirmed that the use of the ER in the physics laboratory generates more motivation towards physical science than in the same physics laboratory using only conventional equipment. This result is in line with the empirical work of Arí s and Orcos [13] on students' motivation and attitude toward STEM. They observed that robotics was able to stimulate more motivation toward these subjects. More recently, researchers [28] also concluded that ER had a positive impact on students' motivation in education in general.
Finally, we explain this increase in learning and motivation by the fact that ER is a technology that is consistent with the students' technological everyday life [29] and allows them to better understand it.

VI. CONCLUSIONS
This study aims to identify the effect of educational robotics on students' motivation and learning of physical concepts, in particular temperature. 90 students in the first year of secondary school belonging to a public school in the city of Meknes-Morocco, participated in this study: 46 in the experimental group and 44 in the control group. Two measures were collected at the end of the task. A performance index resulting from the pre-and post-test correction assessed the learning of the notion of temperature, and a Likert scale questionnaire assessed the student's motivation.
The main conclusion of this research is that the integration of educational robotics into a physical science task is more effective in stimulating student motivation and learning than the same task in a laboratory using traditional equipment.
The main limitations of this research are related to the lack of a representative sample of the population and the lack of control for the novelty effect. Similar studies can be conducted with a large sample and examine the attitudes of students of different levels toward science subjects after the use of ER. So that the results can be generalized. Furthermore, the integration of ER within our task was also punctual, in the sense that robotics was only used as a tool: the building and programming activity was not central to the task, as it could have been if we had favored another approach than learning by doing. As such, a future research perspective would be to integrate robotics centrally into physical science teaching using, for example, problem-based or project-based learning.
Considering our findings, we believe that ER offers educators the opportunity to transform traditional education into a new form of an innovative and technologically coherent learning experience for 21st-century students. This study recommended that physical science teachers integrate educational robots into their teaching activities to improve the effectiveness of student learning and enhance student motivation. It is also recommended that researchers conduct further studies on the integration of educational robots at different educational levels. Additionally, it is recommended to the authorities of the Ministry of Education in Morocco to set up in-service training for the benefit of teachers to qualify them to use educational robots. APPENDIX Appendix A As part of an educational research project, please answer the following questions anonymously and carefully.
□ TRUE □ FALSE 2-The temperature of pure water remains constant throughout the melting process □ TRUE □ FALSE 3-The boiling temperature of pure water at normal atmospheric pressure is □ 90 ℃ □ 100 ℃ □ 104 ℃ 4-The melting temperature of pure water at normal atmospheric pressure is □ -6 ℃ □ 0 °C □ 5 °C 5-The temperature of the saltwater remains constant throughout the boiling process □ TRUE □ FALSE 6-The temperature of the saltwater remains constant throughout the melting process.
□ TRUE □ FALSE 7-Each pure substance has its own temperature of change of physical state.
□ TRUE □ FALSE 8-Each mixture has its own temperature of change of physical state □ TRUE □ FALSE 9 -During melting, the pure body loses heat □ TRUE □ FALSE 10-During solidification, the pure body loses heat □ TRUE □ FALSE Appendix B We invite you to respond spontaneously and truthfully to each of the following statements. This is not an exam. There are no right or wrong answers. It is only intended to find out your opinion about the activity you have just completed.