An Internet of Things ( IoT ) Maker Curriculum for Primary School Students : Develop and Evaluate

It is acknowledged that the Maker movement has raised up a revolution at developing our future manufacturing and economy. So, it is significant that the current empirical studies of Maker Education may have led a brand new way for the competence-oriented curriculum reform. The purpose of this study tends to develop a module course of Maker Education, in which the learning effectiveness can be verified through its teaching/learning process. In this study, the courses have been designed and practiced for 47 students (5th grade of an elementary school), meanwhile the study data have been collected and analysed to be found that the learning effectiveness in the factors of “Theoretical knowledge”, “Practical application” and “Maker’s mindset” all have conformed to or exceeded the expected results. According to the research results, the students have obviously absorbed and internalized the theoretical knowledge of science, technology, engineering and mathematics into the course learning, with the ability to transform the theoretical knowledge into practical operations. In addition, they also have maintained the manufacturer's mindset in their learning production process, with a high interest in acquiring knowledge, and can actively think about the impact of technology at encountering problems.


I. INTRODUCTION
Because of the "Maker" movement, the topic of "do it yourself (DIY)" has regained public attention and been brought to the level of "common creation." Since 2011, Maker-related issues have been increasing year by year, and have shown fruitful results in both curriculum design and academic research [1], which all gave positive affirmations to students' learning situation. The areas of observed learning growth include: Understanding of computer science concepts [2], computational thinking [3], STEM literacy [4] , imagination [5], self-efficacy, scientific identity, and possible selves [6], experience of fun and learning [7], and 3D Design ability [8], as well as positive impact on learning attitudes in computer science, mathematics, and engineering [9]. However, these achievements lack a complete exposition of the core Maker concepts. The indicators of Maker education in elementary schools are including three aspects [10]: 1) Making: possess the theoretical knowledge and apply it in reality; have an appropriate maker's mindset to overcome difficulties in the face of challenges 2) Collaboration: enthusiastically participate in group activities and have positive interactions with peers to fulfill group objectives. 3) Knowledge sharing: be motivated to share knowledge and information and do so in an appropriate way. Maker education has the characteristic of "strengthening the linkage between curriculum and life". In addition to "knowing", learners must have the ability of "using" and maintain their enthusiasm for "thinking". Through "making" activities, learners can be trained to have the ability to adapt to today's life and participate in the future world. The empirical studies of Maker Education at present lead a brand new way in the competence-orientated curriculum reform. Therefore, this study aims to develops a teaching module of Maker Education in elementary school.

A. Maker Educational Curriculum Design Model
"Making" is the core competence of a Maker, and with the progress of science and technology, the encountered problems become more complex, which means that interdisciplinary knowledge is needed for problem solving. According to the empirical research of Papavlasopoulou et al. [11], Maker education is often operated in the STEM, which allows learners to learn how to integrate and apply knowledge through projects. Fan and Yu [12] pointed out that backward design of curriculum and informed design of curriculum are often adopted during operation of the STEM; therefore, the development of this research course adopts the backward design of curriculum, as proposed by Wiggins and McTighe [13], for curriculum planning, which is divided into the following 3 stages: 1) Identify desired results: Teachers should fully understand the connotation and study focus of the selected topics, and establish the desired learning results of the students after completing their studies, including the desired results in cognition, ability, and attitude. 2) Determine assessment evidence: In order to determine whether a student achieves the expected learning results, it is necessary to select or develop appropriate assessment tools in stage 2. 3) Plan learning experiences and instruction: Teachers plan learning content and arrange teaching activities. Overall, the backward design emphasizes the logic and relevance between the objective results, the assessment model, and the curriculum content.

B. Determination of Teaching Topics
The popularity of the Internet of Things (IoT) has made the Internet move from the virtual digital environment to the real physical environment. Qin et al. [14] pointed out that IoT has applications in many fields, such as smart cities and families, environmental monitoring, health care, energy, and commerce. IoT is a large network based on the Internet, and covers all kinds of terminal equipment and facilities. The European Telecommunication Standards Institute [15] defines the development of IoT applications as including the following three areas: 1) Perception layer: sensing environmental data through various sensors; 2) Network layer: receiving data from the device layer, transmitting it to the application layer, and acting as a bridge. 3) Application layer: utilizing and achieving intelligent control and management through cloud computing and large data analysis. In the operation of the perception layer, learners must learn how to make the device have the functions of "identifying, sensing, and controlling" the various status data of the terminal object, and then, transmit the sensed information to the network through the sensing network communication module. In the network layer part, learners must learn how to transform and transfer the centrally sensed information to the application layer. In the application layer, learners can learn cloud computing, big bata, data mining, machine learning, deep learning, etc. When sensing that information is being transmitted through the network to the application system, it is necessary to link a single piece of information on the Internet of Things into a set of networks, conduct further analysis and computing, establish operation models, set automatic handling for specific events, and guide operational decision-making to provide the operation of the whole system. Only in this way can it be a complete Internet of Things.
As the Internet of Things (IoT) is now an important technology trend and highly related to learners' daily life through various applications, such as wearable devices, smart families, environmental detection, etc., it is very suitable to be introduced into the alternative curriculum of elementary schools. Therefore, this study sets the Internet of Things as the curriculum development topic. In addition, Hsu et al. [16] suggested that a interdisciplinary instruction should be adopted to enable students to deepen their understanding of interdisciplinary knowledge and foster their interest in science, technology, engineering, and mathematics through computational thinking and data analysis in various fields. Therefore, this study takes the Internet of Things as the teaching theme and integrates learners' life situations into the curriculum design. However, due to the limited knowledge of primary school students, it is difficult to carry out in-depth work development. Hence, this study limits the scope of topics to allow learners to have a preliminary cognitive concept of the Internet of Things, try to connect the device layer, network layer, and application layer through practice, and then, design simple IoT products.

C. Plan Learning Experiences and Instruction
Based on the indicators of Maker education in elementary schools, as proposed by Yeh et al. [10], this study proposes the desired learning results for students to gain "making" ability in their life, including 3 factors and 10 indicators: 1) The theoretical knowledge factors:  Knowledge of science: be able to comprehend scientific principles in daily life;  Knowledge of technology: be able to comprehend the conceptual and procedural knowledge of technology;  Knowledge of engineering: be able to comprehend engineering principles applied in the making of products;  Knowledge of mathematics: the mathematical abilities to analyze, count, compute and measure 2) Practical application factors:  Hands-on movements: be able to operate, use and maintain tools  Design and planning: design targeted, functional and attractive products using design rules;  Conversion and application: be skillful in using theoretical knowledge to design and make products; 3) The Maker's mindset factors:  Learning interest: be passionate about making and become engaged in it;  Active thinking: be able to solve problems with a positive attitude.  Influence of technology: be active to understand the interaction among technology, individuals and family life. According to the desired learning results, the content of this curriculum is planned as Table I, which is divided into hands-on activities and personal project. The first 11 weeks of the curriculum are hands-on activities. Through the analysis of life cases, students can learn how to edit programs, assemble hardware, and design and plan the products. The hands-on activities are implemented under the principle of the 6E teaching strategy, as proposed by Burke [17]: "Engage, Explore, Explain, Engineer, Enrich, and Evaluate", in order to cultivate students' ability to explore knowledge. Table II illustrates the comparison between the indicators of this curriculum and the 6E strategy, and takes the example unit "Hands-on activities (III)-Ultrasound Sensors: Technological Applications of Acoustic Waves" for implementation in the 10th week. Through the practice of the ultrasonic sensors, students begin to carry out "assemble hardware" and "programming", which combines curriculum knowledge and practice: 1. Installation of Ultrasound Sensor Circuit 2. Algorithms for distance calculation of ultrasound sensors 3. Compiling program for distance calculation of ultrasonic sensors

2-2 Engineer
Teachers demonstrate the principle of "Engineer" by first showing the design direction of the product, such as "what conditions the intelligent packaging sensors should have", guiding students to tell the purpose of the product, and then, showing the impact of the appearance of IoT on the product functions.

2-3 Enrich
According to the above design and planning, the teacher assists students to make products, such as producing an intelligent reverse sensor with the distance display function by combining it with LED or a screen display.
Through evaluation, help students to understand their learning effectiveness and give suggestions on how to optimize the product.
In the last three weeks of the curriculum, students apply what they have learned to make products. Personal project adopts informed design of curriculum [18], which means that students will plan a series of courses for knowledge and skills building before personal project. Compared with the objective model, the informed design of curriculum attaches greater emphasis to the spirit of inquiry and analysis, and ensures that the process of personal project displays the innovation and application of students' high-level thinking ability. Just like the hands-on activities, the operation follows the 6E process, except that the original semi-open questions with teacher assistance are changed to open questions, and emphasis is placed on the six steps of the engineering design principles, including: 1) Task definition: Identify the purpose and function of the product; 2) Development project: Plan hardware and programming; 3) Predictive analysis: Assume possible situations and problem shooting; (4) Prototype modeling: Practice the above; 4) Test and revision: Test whether the product works; 5) Optimization: Observe other works and make adjustment on one's own works. The core hardware device used in this curriculum is Webduino Smart, as developed by KING KIT TECHNOLOGY CO., LTD, based on Arduino, and embedded in the Esp8266 module. This study used Webduino Blockly software, as proposed by the company, to modify the Blockly program code.

III. RESEARCH DESIGN AND IMPLEMENTATION
The subjects of this study are 47 fifth-grade students in a elementary school in Kaohsiung, who had the experience of learning Scratch in the fourth grade, and thus, possess the basic concepts of computational thinking, such as: decomposition, which refers to decomposing a task or problem into several steps or parts, and pattern recognition, which refers to predicting the law of the problem and determining the test model. The course is planned to last for 14 weeks, with 80 minutes of course content each week.
Data collection was carried out by using the test method and the observation method. The test method used standardized testing as a tool for data collection [19]; the observation method refers to the systematic observation, recording, and objective interpretation of the samples' behavior in natural and controlled situations in a planned manner.
The tools used in the test method are the "IoT Conceptual Knowledge Test" and "Maker's mindset scale". The "IoT Conceptual Knowledge Test" contains five questions of the scientific dimension, seven questions of the technological dimension, five questions of the mathematical dimension, and three questions of the engineering dimension, for a total of 20 questions, all of which are single-choice, with 5 points for each question, and a full score of 100 points. The average difficulty index of this test is .66 and the average discrimination index is .35, which shows that the questions are moderately difficult and have good discrimination [20].
The "Maker's mindset scale", which is divided into the three factors of "Learning interest", "Active thinking", and "Influence of technology", refers to the results of Yu, Lin, Han, and Hsu. [21]. This questionnaire adopts the scoring method of a Likert seven-point scale, where the higher the degree of identification, the higher the score. Regarding scale preparation, a questionnaire survey was conducted with 5 experts in technological education to establish content validity. If the cumulative percentage of items judged as "retain" or "modify" was more than 80%, the items were retained [22]; otherwise, they were deleted. Finally, all items were retained.
The tools used in the observation method are the "Hands-on activities worksheet" and "Project evaluation rubric", which refer to the framework proposed by Fan [23]. These two tools are composed of three items: "Hands-on movements", "design and planning", and "Conversion and application", which are rated by a scale. The scoring rate of each item is obtained by dividing the scores of the individual sub-items by the total score of the item, and then, multiplied by the percentage (scoring rate = score / total score * 100%). The scoring rate is presented by a five-point scale in class intervals, with five points representing that 80-100% of the item can be achieved, four points representing that 60-79% of the item can be achieved, three points representing that 40-59% of the item can be achieved, two points representing that 20-39% of the item can be achieved, and one point representing that 0-19% of the item can be achieved.
Multiple Assessment is adopted to evaluate the learning results. In addition to using different tools, the evaluation is divided into formative evaluation and summative evaluation according to the time of evaluation. Formative evaluation represents the effect of students' short-term memory, while summative evaluation can explain whether learning results can be internalized into long-term memory. This study analyzed the IoT conceptual knowledge test through descriptive statistics, and the results illustrate the overall learning effectiveness of the students. The questions with a higher error rate were analyzed according to the myths of the students. Both the hands-on activities worksheet and the project evaluation rubric adopt the form of a five-point scale. Through one-sample t-testing, the three points are used as the verification value to analyze whether the students' learning results have achieved the expected targets of this study. Finally, the data obtained from the Maker's mindset scale are processed using Repeated Measured ANOVA to test the changes of the Maker's mindset factors after students participate in the course. The curriculum dimensions, evaluation tool/items, and data analysis method are shown in Table III.

A. Learning Effectiveness in the Theoretical Knowledge Factors
Formative evaluation of the theoretical knowledge factors was carried out through the basic conceptual item of the hands-on activities, and the analysis results are shown in Table IV. Except for the scores of the basic concepts of Hands-on activities (III), which have no significant difference from the verification values, the scores of the learning effectiveness of all other items are higher than the verification value of 3, which shows that students have knowledge of the relevant IoT theoretical concepts provided in the curriculums of the hands-on activities. The summative evaluation of the theoretical knowledge factors adopts the IoT conceptual knowledge test with an average score of 67.39, a standard deviation of 2.10, and a total score of 100 points. If 20 points are used as one class interval, by converting the class intervals according to the conceptual analysis of the rating scale, it can be found that, as a whole, the students have good scoring performance in the IoT conceptual knowledge test. Among the class intervals scores, 48% of the students had 60-79 points and 28% had 80-100 points, for a total of 76%, which shows that most students understood and absorbed the theoretical knowledge factors of the textbook content, and internalize it into long-term memory. The distribution of class intervals is shown in Table V. Analysis of the four indicators of the theoretical knowledge factors found that students performed poorly in mathematical knowledge (as shown in Table VI), which shows that students did not have good mastery of abstract reasoning or application abilities; for example, Question 18 is on the calculation of acoustic velocity, and the correct answer fill rate is only 26%. Therefore, teachers must use specific examples to interpret the concept of velocity, and clarify that "velocity" refers to the distance the object moves in a unit time, in order to explain the relationship among velocity, distance, and time.

B. Learning Effectiveness in the Practical Application Factors
Formative evaluation of the practical application factors was carried out through the Hands-on movements, Design and planning, and Conversion and application items in the evaluation of hands-on activities worksheet. The scores in all items are significantly better than 3 points, which means that most students mastered the skills of practical application, and the analysis results are shown in Table VII. Summative evaluation of the practical application factors was carried out through the project evaluation rubric. Students obtained an average score of 4.71 in hands-on movements items, 4.07 in design and planning items, and 4.04 in conversion and application items, which means that students can convert theoretical knowledge and apply it into practice. The scores of the items and the number percentages are shown in Table VIII. In the hands-on movements items, 80.4% of the students scored 5 points, representing that 80% of the students are very skilled in the use of tools. Secondly, in the design and planning items, 47.8% of the students scored 5 points, representing that nearly 50% of the students can design works by themselves for specific purposes, and can conceive that, in order to achieve such purposes, they must determine what functions a work must have, and what hardware and program these functions must cooperate with. In the process of making a product, they can incorporate the potential problems and consider how to solve such problems. Regarding Conversion and application items, 37% of the students scored 5 points, representing that about 40% of the students can convert their design into products through the Making process. The students who scored below 3 points account for 28.2%, which shows that these students did not experience a smooth Conversion and application process; although they have the basic theoretical knowledge and ability, there is a considerable gap in the actual application.

C. Learning Effectiveness in the Maker's Mindset Factors
The Maker's mindset factor is measured by the Maker's mindset scale in the 5th, 8th, 11th, and 14th week of teaching. First, overall descriptive statistics analysis was carried out, and the results show that, the average scores of the three indicators are more than 6 points, representing that most students can retain their learning interest after taking this course, engage in active thinking when facing problems, and realize the influence of technology. The analysis results are shown in Table IX. The Mauchly's sphericity test was used to test whether the data to be analyzed conformed to the spherical hypothesis, and the results show that the Mauchly's sphericity test of learning interest, active thinking, and Influence of technology are 0.01, 0.11, and 0.001, respectively, while their significance (P values) are less than 0.001, which does not conform to the spherical hypothesis. Thus, epsilon was used to make corrections, the Huynh-Feldt (H-F) values were used to correct the F test, and the results show that the F values of all the three indicators did not reach significance levels, indicating no significant difference in the four tests. Moreover, their Maker's mindset were maintained at a certain level, meaning they did not lose interest in the course due to increased course difficulty or reduced assistance from the teacher, and the results are shown in Table X.

V. CONCLUSIONS
The results indicate that the Maker curriculum has an impact on students' learning effectiveness in the following factors: "Theoretical knowledge", "Practical application", and "Maker's mindset".
1) Theoretical knowledge: The students could internalize the theoretical knowledge proposed in this study, and they could achieve the desired learning effectiveness in both formative evaluation and summative evaluation, which shows that they have a good understanding of the scientific knowledge, technological knowledge, engineering knowledge, and mathematical knowledge presented in the curriculum. Meanwhile, they could also engage in effective "Conversion and application" of the practical application factors during product production, which shows that students have a holistic concept of theoretical knowledge. 2) Practical application: The students had the ability of practical application to transform and apply the theoretical knowledge into practice. Their effectiveness in the formative evaluation of the three indicators of "Hands-on movements", "design and planning", and "Conversion and application" were significantly better than expected, showing that in the learning process of the three units, with appropriate guidance by teachers, students can have the ability to design products, skillfully use tools, and transform and apply what they have learned in production. In addition, in the summative evaluation, the teachers gradually gave less assistance and added contents to the dimensions, thus, greatly increasing the difficulty of curriculum content; however, the students' average scores in "hands-on movements", "design and planning", and "Conversion and application" items were all higher than the default verification values. More than 80% of students received 5 points in the "Hands-on movements" item, which shows that most students became proficient in using tools. In "design and planning" and "Conversion and application", about 70% of students could make the products independently or with only a few tips from their teachers. However, due to the expanded depth and width of curriculum difficulty during personal project, about 30% of students required more specific assistance from teachers to design and make the products. 3) Maker's mindset: The students had the Maker's mindset, maintained their learning interest, engaged in active thinking when facing problems, and realized the influence of technology: their scores in "learning interest", "active thinking", and "Influence of technology" are all more than 6 points, meaning they maintained a high level of effectiveness even with the gradual increase in curriculum difficulty, which shows that they maintained curiosity and interest in learning the course contents, and could actively consider how to overcome and solve the difficulties or problems they encountered. In addition, in the process of practice, they could realize how technology, and the products they designed, can change their lives in the future. Maker courses still need to emphasize interdisciplinary integration and integration into real-life application scenarios. The interdisciplinary teaching mode can enable the students to develop and understand cross-disciplinary knowledge. During the learning process, they need to constantly think and explore in their learning activities to maintain their curiosity and motivation. This study only provides a learning alternative, so that Making can have an impact on their learning. It is expected that a lot of continuous research is needed to consolidate this teaching strategy and become a part of basic education.