HOTS-Oriented e-Project-Based Learning: Improving 4C Skills and Science Learning Outcome of Elementary School Students

 Abstract —Overcoming elementary students learning loss during the pandemic Covid-19 through implementing an online project-based learning model has been taken into account. The study aimed to determine the effectiveness of the high-order thinking skills oriented online project-based learning (HOTS-oriented E-PjBL) tools for improving 4C skills and science learning outcomes of fourth-grade elementary students. A quasi-experimental design with a post-test control group design was implemented. The samples were 301 pupils consisting of 149 experimental class students and 152 in the control group. The learning outcome data were collected through an essay-type test. The observation sheet was used to measure 4C’s skills. The data were analyzed by using the inferential statistical analysis of MANOVA. The results showed an effect of the HOTS-oriented E-PjBL tools on improving 4C skills and science learning outcomes simultaneously and partially. Another finding was that the HOTS-oriented E-PjBL tools affected 21st-century skills more than the science learning outcomes. Thus, HOTS-oriented E-PjBL is recommended to improve students’ 4C skills and science learning outcomes in elementary schools.


I. INTRODUCTION
The Covid-19 outbreak made it necessary to implement large-scale social restrictions imposed by Government Regulation Number 21 of 2020 that required everyone to work from home and study at home during the Covid-19 pandemic and caused the implementation of teaching shift [1][2][3]. Online learning is implemented in synchronic and asynchronous ways; the students and the teachers and students carry it out in their respective places [4][5][6]. Students can discuss with teachers and other students, teachers can send assignments, and evaluate virtually with video conferencing, google classroom, google form, google meet, WhatsApp, telegram, and other tools that can be used in the learning process. This feature allows HOTS-based learning to be carried out even though learning is virtual/online [7]. The teacher and the students must be able to use technology when they are engaged in online learning. A conducive learning atmosphere is largely influenced by the ability to use technology to ensure better social interactions [8]. Additionally, as educators, teachers must understand the Manuscript  students' financial conditions to support online learning [9]. Every effort is made to improve the effectiveness of online learning by collaborating online learning with innovative learning. One creative learning model is project-based learning (PjBL) [10]. Project-based Learning (PjBL) is a learning model which is highly recommended to be used by the teachers to support the implementation of the scientific approach in the 2013 curriculum [11]. Students are led to learn through direct experience through this learning model, which is also known as experiential learning. This is because PjBL involves students solving real problems found every day, in which the solutions are expressed in the form of projects. The students will become familiar with developing their high-level thinking skills through the learning focused on problems and projects. In this way, the students given the opportunity to express their ideas about the project they are going to work on. This allows them to develop high-level thinking skills. PjBL learning effectively develops higher orderthinking skills. This is because the PjBL model allows students to participate actively in the learning processto develop critical thinking, problem-solving, and creative thinking skills. The three thinking skills are indicators of higher order thinking skills [12][13][14][15]. This statement is supported by research findings showing that the PjBL model effectively develops problem-solving ability and students' critical thinking skills in essential statistical learning [16]. Several studies have shown the advantages of PjBL compared to other learning models. It effectively improves critical thinking skills and soft skills. The PjBL-STEM model can be used alongwith other materials with STEAM characteristics. Spectra-plus-assisted STEAM-based PjBL learning increases scientific critical thinking skills [17]. The development of local wisdom-based PjBL models enhances the student's ability to solve environmental problems [13,16,18]. So, in other words, learning with the PjBL method positively influences the ability to think critically, solve problems, and achievea high learning outcome. Furthermore, PjBL contributes to the development of 21st-century skills.
The US-based Partnership for 21st Century Learning states that one of the most critical skills mastered and taught is learning and innovation skills, known as "The 4Cs", critical thinking and problem solving, creativity and innovation skills, collaboration, and communication [19][20][21]. These skills allow students to face challenges in everyday life [22]. The 4C skills are compulsory and are taught at all levels, especially in primary school education in the field of study and the theme of the 21st century [23]. It can also be said that the skills of the 21 st century are crucial to be mastered by everyone to succeed in facing challenges and problems in the 21st-century life.
Communication skills are essential for students and are the primary requirement everyone must master. Communication skill is a person's ability to convey or express their thoughts to others [24]. Effective communication will greatly help children in the process of interpreting situations and improving relationships with others. With their ability in effective communication skills, the children can also develop their social and emotional skills [25]. This means that the communication skills that students must acquire must be good and effective. Good communication skills will make students know and build good relationships with others and the outside world.
Collaboration is the ability to participate in any activity to foster relationships with others, mutually value relationships, and build teamwork to achieve common goals. In the learning process, teachers should create a situation where students can learn together or in groups. Students with collaboration skills will contribute to the learning process for other students. Collaboration skills will allow the students to improve learning outcomes This is because of the cooperation among peers. Peers are related to each other, which inspires a desire to help so that they can create good interaction and cooperation in learning. Of course, this will demand learning that can actively involve students learning in small groups and developing thinking skills.
Critical thinking and problem-solving are among four deep learning skills identified as critical to the development of individuals who are ready for the challenges of today's globally interconnected world. Critical thinking is self-thinking that generates new and innovative ideas and solves problems, reflects critically on learning experiences and processes, and makes effective decisions. To develop critical thinking and problem-solving skills the learning process should allow students to think critically by connecting learning with contextual problems in everyday life to develop critical thinking and problem-solving skills.
Creativity and innovation skills are the skills that someone must have to survive and compete in the 21st century. Students with good creativity and innovation skills will demonstrate flexible thinking, open-mindedness, and good behaviour. Today, creativity and innovation cannot be separated from education. Therefore, learning must create conditions for students to be creative and innovative. Teachers become facilitators in accommodating the results of creativity and innovation skills developed by students.
To answer the challenges of learning in the 21st century, the government has made various efforts to improve the quality of education in Indonesia to create people who can compete in the 21st century and have the 4C capabilities. These efforts include revising and updating the curriculum. Currently, the curriculum used in the learning process is the 2013 curriculum (K-13). The learning process that is carried out has emphasized the learning process in the 21st century. 21st-century learning requires teachers to change from old learning approaches that are less relevant to the 21st century to learning techniques relevant to the 21st century. Teachers in the 21st century must become experts in searching for information with students, know how to collaborate, and guide students to discover discoveries in every learning process. Learning in the 21st century is more focused on students, where students play an active role in finding and building their knowledge. Through the teacher's guidance, the students are required to develop the ability of HOTS [24,26]. One of the suggested learnings is PjBL because this learning model allows students to develop HOTS. However, it is undeniable that there are still obstacles that teachers and students must face, such as the unavailability of the internet and mobile phones, communication that does not work well, parents' difficulty accompanying their children, and the teacher's limited ability to utilize information and communication technology.
This condition corresponds to the result of the initial analysis conducted, where the results show that teachers are facing problems in designing PjBL learning during the pandemic. The results, among others, are as follows:1) 54% of the teachers have difficulties in determining or creating the PjBL format; 2) Teachers have problems in compiling student worksheets for PjBL-based learning (67%); 3) Teachers have difficulties in developing tools and materials that are suitable for projects in an online learning environment(58%); It is difficult for teachers to incorporate the learning steps into the lesson plan according to the PjBL syntax (61.3%); 5) Teachers do not have a clear understanding of the PjBL model (69.1%); 6) Teachers have difficulty with defining syntax in project-based e-learning design (61.3%); 7) It is difficult for teachers to develop projects that need to be given to students when learning online (72.6%); 8) It is difficult for teachers to develop teaching aids/materials suitable to the needs of the PjBL model (66.1%); 9) Teachers encounter barriers in determining time allocation in the PjBL learning design (66.1%); and 10) Teachers have difficulty in determining the number of groups that do PjBL learning (61.3%). So, it can be said that teachers have obstacles in designing the PjBL learning format due to several factors, including the fact that teachers do not understand PjBL learning and how to apply PjBL in the online learning process. To overcome these problems, there needs to be a solution that suits the needs of the teachers in the education course. The solution presented in this research is to develop a HOTS-oriented E-PjBL tool [27,28].
This study aims to determine the effectiveness of the high order thinking skills oriented online project-based learning (HOTS-oriented E-PjBL) tools for improving 4C skills and science learning outcomes for fourth grade elementary students. The tools developed in this study were the tools that use the online PjBL model designed to be implemented in the HOTS-oriented E-PjBL to facilitate the development of the students' 21st-century skills. This model was selected for several reasons [29,30]. Firstly, the PjBL typically is one of the models suggested in the 2013 curriculum that can facilitate students' high-level thinking skills and encourage students to produce contextual work. Secondly, the PjBL can spur students' competitive power. The direct experience causes students' memories of materials to last longer, hones students' collaborative skills in solving problems and completing projects in groups, and improves the students' social interactions in groups. Thirdly, the tools developed used ecosystem materials because, by the initial study of the students' understanding of ecosystem materials, the students had not yet shown the desired level of completion [31][32][33]. In addition to moving from these conditions, the consideration was that ecosystem material studies the occurring relationships in the environment around the students [34]. PjBL would not cost much because the project could be done using tools and materials in the environment around the students [35]. Fourth, the materials were packaged online, considering that the situation and conditions in this pandemic period preclude face-to-face learning. These online learning tools provide an opportunity for parents to engage directly in their children's learning activities to know and control their learning activities. This is not only true in the current pandemic time, but with the development of technology in the future, learning is likely to continue to be implemented online. The combination of online education and HOTS-oriented PjBL requires the support of tools that can facilitate, monitor, and evaluate effectively so that online PjBL can run effectively and achieve its goals optimally [36][37][38]. HOTS-oriented online PjBL tools can encourage parents to play an active role in facilitating, guiding, monitoring, and evaluating learning processes and learning outcomes in online learning. The tools developed are valid and practical with excellent categories. The next stage is the test stage of the effectiveness of online PjBL learning with a HOTS-oriented network on 21st-century skills and science learning outcomes [39][40][41][42].

II. METHOD
This study was quasi-experimental research using a post-test control group design [43]. The study's process was grouped into experimental classes and control classes. The experimental group was given a pre-treatment of online PjBL learning or HOTS. The control group was taught the usual way without using the PjBL learning model. The online learning activities were done by developing the learning implementation plan (LIP). The learning activity tools varied: Google meet, WhatsApp Group (WAG), Zoom cloud meetings, and combinations thereof, according to the varied school needs. The learning activities were also combined by using the WAG class. The learning activities were completed by the students enthusiastically. However, not all of the students could join online learning for various reasons, including the limitation of the tools the students had and network interruption.
Post-tests were given to experimental and control groups to determine the differences in their outcomes of the 4C skills and their science learning outcome. The data obtained in this study were 1) the 4C skills (Y1) of the students who were taught with HOTS-oriented E-PjBL tools; 2) the 4C skills (Y1) of those who were taught without the HOTS-oriented E-PjBL tools; 3) science learning outcome (Y2) of the students who were taught with HOTS-oriented E-PjBL tools, and 4) the science learning outcome (Y2) of the students who were taught without using HOTS-oriented E-PjBL tools. The trials of the learning tools were conducted at five elementary schools in Cluster III of Buleleng District of Buleleng Regency, that is; 1) SDN 3 Banjar Jawa; 2) SDN 1 Banjar Jawa; 3) SD Mutiara; 4) SDN 1 Astina; and 5) SDN 1 Paket Agung. Two classes were taken from each elementary school: one as an investigational group and the other as a switch group. The empirical determination and control classes were chosen randomly by lottery. The random sampling result showed that the number of students in the experiment group was 149, while that of the control group was 152.
In this study, the data collection was done using test and observation methods. The test was used to measure the students' learning outcomes. The science learning test was made from 9 indicators and then developed into 30 cognitive levels C4-C6. The learning outcome test instrument was compiled by the researcher with these steps: basic competencies analysis, preparation of the problem grid, the creation of question items, expert validity test, revision, trial, analysis of the trial results, and development of the final test that was ready to be used to collect science learning outcomes data. Indicators of science learning outcomes are presented in Table I. Meanwhile, the 21st-century skill instrument used was an observation sheet and essay test with assessment rubrics covering all four aspects of 21st-century learning and innovation (4C) skills. Each of the skill aspects is presented in Table II. After developing the learning outcome test, the instrument was tested for validity, reliability, discrimination power, and difficulty level. By presenting a discourse, students can infer the use of animals according to the type of food. 3. Distinguishing the types of animal life cycles.
By presenting questions and images, students can distinguish three animal life cycles. 4. Analyze the food chain contained in an ecosystem. By presenting a food chain and food web, students can analyze the food chain contained in one ecosystem. 5. Analyze the relationship between the food chain and the flow of energy in an ecosystem Students can analyze the relationship between the food chain and the energy flow in an ecosystem by presenting a discourse. 6. Comparing the symbiotic forms that occur in the surrounding environment.
By presenting several symbiotic images, students could compare the shape of the symbiosis. 7. Showing the cause of environmental changes on the sustainability of food webs By presenting a list, students can show things that can cause changes in the food web. 8. Showing the consequences of environmental changes on the sustainability of food webs By presenting a picture and description, students can analyze the consequences of environmental changes on the sustainability of food webs. 9. Evaluating human activities that may affect ecosystem balance.
By presenting a paragraph, students can evaluate human activities that can affect the balance of the ecosystem. 1. Able to express the lack of ideas that have been developed 2. Able to find ideas to make products from materials from the environment. 3. Able to act on creative ideas to make a real and beneficial contribution to the group and yourself. Implementing innovation 1. Able to realize the idea into a product both independently and in groups. 2. Able to develop ideas into work.
The content validity test was determined with Gregory's validity test. The mechanism for calculating the values is as follows: 1) experts rate each instrument; 2) ratings are grouped into less relevant and highly relevant; 3) the results of the expertise are tabulated in the form of a matrix; 4) cross-tabulation is performed between two experts; 5) Content validity. Based on the validity test of the contents of the science learning achievement test that had been done, the coefficient of validity of the science learning achievement test was 1.00. The validity of the student's science learning achievement test contents in this study was very high. To measure the internal validation of the science learning achievement test consistency, the biserial point correlation coefficient (r pbi ) formula was used. Furthermore, the r pbi (real) value was compared to the constant 0.30. If r pb ≥ i 0.30then the question item was valid. Conversely, if the r pbi <0.30, the question item was invalid. The science learning achievement test consisted of 36 questions from 9 indicators set so that four problem points represented each indicator. The test was piloted on 98 respondents who had obtained science material listed on the grid of subject items. The test analysis results showed that of the 36 questions tested, four items were not valid; they were item numbers 11, 13, 22, and 26. The four items represent different indicators. Based on the discussions with teachers at the school where the study took place, the science learning outcome test used 30 questions. A reliability test was carried out on the valid items only. Thus, a reliability test could be completed after the validity test.
The reliability test that focuses on dichotomy and heterogeneity was determined by the Richardson formula . Based on the reliability test of science learning outcome that had been carried out, it has a reliability coefficient of 0.794, so the reliability of the science learning outcome test in this study falls into the high category. Because the reliability coefficient of the science learning outcome in this study is above 0.60, this science learning outcomes test was suitable for use in this study. Item discrimination is the ability of an item to distinguish between highly capable students (intelligent objects) and low capable students. The way to determine the upper group from the lower group is to take 27% of the sample numbers each for the upper and the lower groups. The process of determining the upper and the lower groups of students is by sorting each student's score from the highest to the lowest. Then 27% of the upper and 27% of the bottom groups are taken. The analysis result showed that 22 items of the questions fall into the category of good discrimination power and 10 items into the category of enough discrimination power. The difficulty level can be viewed as students' ability to answer a test. The adequacy of the test items' difficulty level can be known from the number of items representing the item's difficulty level. The figure that indicates the difficulty or ease of an item is called the difficulty index. The magnitude of the difficulty index (DI) is categorized as follows, the values of 0.00≤DI≤0.30 are difficult, 0.30DI0.70 values are moderate difficult, and DI≥0.70 are easy. This difficulty index shows the difficulty level of the item. An item with a difficult index of less than 0.30 indicates that the item is difficult. On the contrary, an index of more than 0.70 suggests that the item is easy. Based on the level of difficulty test of the science learning outcome test that had been developed, it was found that all items (32 items) of the science learning outcome test in this study fall into the medium category.
The data of post-test scores were statistically analyzed descriptively and inferentially using SPSS 25.0 for Windows. Scores found on statistical tests are the mean, standard deviation, maximum and minimum scores. Inferential statistical analysis with the MANOVA test for post-test data. Before performing the test, the standardization and uniformity tests are performed first. The normality test used was the Kolmogorov-Smirnov test and the homogeneity test used was Levene's statistics. The same is true for the MANOVA test. Before the MANOVA test, preliminary tests were performed, including the normality test using the Kolmogorov-Smirnov test and the homogeneity test using Levene's statistic and the covariance matrix equality test box, and linearity was checked to determine if there is a linear relationship in each dependent variable being analyzed. MANOVA testing and claims testing were performed using SPSS 25.0 for Windows.

III. RESULTS AND DISCUSSION RESULTS
Descriptively, the research results showed that implementing HOTS-oriented E-PjBL tools affected the increase of 4C skills and science learning outcomes of ecosystem learning subject matter. This was indicated by the significant difference in mean score of the 4C skills of the students taught with the HOTS-oriented E-PjBL tools and those taught without the HOTS-oriented E-PjBL tools, where the mean score difference was 13.30. In contrast, the mean score of the students taught by learning using the HOTS-oriented E-PjBL tools was greater than those taught without using the HOTS-oriented E-PjBL tools. Likewise, the mean score of the student science learning outcomes taught using the HOTS-oriented E-PjBL tools was higher than that without the tools. This was indicated by a score difference in the mean score of 6.27. More complete results are presented in Table III. Based on the analysis results, it can be said that implementing of the HOTS-oriented E-PjBL tools effectively improves 4C skills and science learning outcomes. Another finding is that learning using the HOTS-oriented E-PjBL tools has a greater effect on 4C skills than learning without implementing the HOTS-oriented E-PjBL tools. This can be seen from the difference in the mean score for each variable, as depicted in Table III.  The results of normality test show that the study results come from normally distributed data. It can be seen from the significance values. The results of normality analysis are presented in Table IV. The results of homogeneity analysis show the same meaning that the study results come from homogeneous data sets. This can be seen from the significance values. Each test showed a value greater than 0.05. sig. value. Levene's equality test is 0.93 for the 4C skills and sig. value for the science learning outcomes was 0.57. Meanwhile, the result of the uniformity test with covariance matrix equality test Box shows sig. = 0.5 and F = 0.88. This means there is no linear relationship between the data on science literacy and creative thinking skills. Prerequisite tests for MANOVA analysis were completed. The obtained data are normally and homogeneously distributed, and there is no linear relationship between the variables so hypothesis testing can be performed with MANOVA. The results of the analysis are presented in Table  V and Table VI. The analytical results in Table V show that, based on Pilae Trace, Wilks' Lambda Hotelling's Trace, and Roy's Largest Root, the coefficient F = 3650.07 for sig. = 0.00. This means that there is a simultaneous difference in the 4C skills and science learning outcomes between students who learned with the HOTS-oriented E-PjBL tools and those who learned without the HOT-oriented E-PjBL tools, or regular learning. The analysis results in Table VI show a significant difference in the 4C skills between the students who learned with the HOTS-oriented E-PjBL tools and those without the HOTS-oriented E-PjBL tools with F =430.46 and sig. = 0.00. There was a significant difference in science learning outcomes between the students who learned with the HOTS-oriented E-PjBL tools and those who learned with the conventional tools with F = 342.90 and a sig. = 0.00.  The results show that the HOTS-oriented E-PjBL tools effectively improved 4C skills and science learning outcomes for elementary school students. This can be seen from the difference in the mean scores, where the mean score for the experimental group, in this case, the students who were taught through the HOTS-oriented E-PjBL tools, was greater than that of those who were taught conventionally. This cannot be separated from how this learning was delivered. The HOTS-oriented E-PjBL process was actualized by implementing the tools following the needs and characteristics of the students. In addition, the learning process using the developed learning tools of the HOTS-oriented E-PjBL was certainly more exciting because the developed learning tools were compatible with the elementary school student's characteristics, where the elementary school students are at a concrete cognitive level. According to Piaget, each child goes through the stages of development in the same order. The order never changes from the sensory-motor stage, pre-operational, concrete, and formal thinking, however, only the speed is different. Therefore, the learning process will create a more interesting and fun learning atmosphere when the teacher pays attention to this. The developed HOTS-oriented E-PjBL tools can effectively improve 4C skills and science learning outcomes because the steps of the E-PjBL facilitate the school student learning processes.
The developed HOTS-oriented E-PjBL tools have interesting designs similar to the students' characteristics. The developed learning tools have exciting designs with pictures that suit the learning subject matter. Furthermore, the developed learning tools follow the students' characteristics at the concrete operational stage. The pictures are those of the objects available in the students' environment. Appropriate picture quality can support optimal learning outcomes and broaden students' insights. In addition, exciting pictures will make the students more motivated in the learning process with the result that when students are motivated in the education process, they will learn actively. Developing learning tools is effective because it provides opportunities for the students to learn from the problems they face in their daily lives. The pupils are interested and active in the process of learning, which deals with daily problems. Students' engagement in the learning process will positively affect their learning outcomes. What follows is a detailed discussion of the findings of this study.
First, there are significant differences in 4C skills among students who have learned with the HOTS-oriented E-PjBL tools and those who learned with a conventional tool. The difference in the 4C skills of students was caused by the learning process carried out in the HOTS-oriented E-PjBL tools that provided opportunities for the students to learn more actively by noting that the learning syntaxes were presented in the learning tools. In addition, the learning process was focused on the problem-solving process. PjBL engages learners with real issues found every day to be solved in learning, where the solutions found are expressed as projects. The learning process focused on problems and projects will familiarize students with their high-level thinking skills. E-PjBL activities build HOTSs thinking skills effectively. In addition, the tools in the form of a lesson plan, teaching materials, teaching media, student worksheets, and learning evaluations are all focused on HOTS. The students will get used to doing high-level thinking activities with such learning.
The learning process with HOTS-oriented E-PjBl tools provides the students with problems related to the ecosystem. The students solve these given problems with parental guidance. The students began to find the solutions that could undoubtedly be used to solve the problems given. In designing a solution to a given problem, the students started to analyse and evaluate the materials relevant to the problem given. Analysing, evaluating, and designing are the characteristics of critical thinking skills. The ability to think critically is the ability to analyse facts, convey ideas, maintain opinions, make comparisons, draw conclusions and conduct an evaluation of the arguments given, and the ability to solve problems. Critical thinking is a person's cognitive ability to express things confidently because it is based on logical reasons and strong evidence. Critical thinking skills provide opportunities for learners to use information sources to produce solutions and provide opportunities for learners to build relationships. Critical thinking skill is an important learning outcome in education [44]. Critical thinking skill is related to the ability of learners to deal with everyday problems [45].

IV. DISCUSSION
The findings of this research are that the implementation of the developed HOTS-oriented E-PjBL tools can improve not only the 4Cskills of elementary school students but also simultaneously enhance the students' science learning outcomes. Those can be explained as follows.
First, the developed HOTS-oriented E-PjBL tools can improve the elementary school students'4C skills, namely critical thinking and problem solving, creativity and innovation, collaboration, and communication skills, creativity and innovation skills are the skills that one has to possess to survive and compete in the 21st century; the students who have good creativity and innovation skills will demonstrate flexible thinking, open-mindedness, and have good behaviour. Creative thinking is a thinking process that can generate many possible responses. Creative thinking includes many high-level thinking skills such as analysis, testing, disruptive communication, and scientific processes. Creative thinking involves flexibility, fluency, novelty, and refinement It also includes divergent and convergent thinking. Creative thinking skills can train students to develop multiple ideas and perspectives, ask questions, recognize the truth about their point of view, and allow students to open up and respond to different points of view. Well-developed creative thinking skills impact learning outcomes. Students with high creative thinking skills will have good learning outcomes. Students can develop their creativity through exciting and varied activities, especially thinking creatively. This diverse and exciting activity is explored in the developed lesson plans. So, it can be said that this learning process can increase creativity skills as it is supported by lesson plans, learning media, and teaching materials designed in line with the HOTS. In the learning process with the HOTS-oriented E-PjBL tools, the students were also accustomed to doing the learning process with peers [46][47][48]. In this case, the students were familiar with learning in a group and building good peer interaction relationships. The students developed the ability to cooperate well, adjust themselves to their peers, and appreciate the group members' contributions. So, in short, the HOTS-oriented E-PjBL tool provides learners opportunities to perform activities involving students with their peers, both in group learning and individual learning. Students are familiar with activities like interacting, working together, and appreciating their friends so their collaborative abilities will develop well. This statement confirms the research that states that project-based learning can improve students' collaboration skills. To collaborate means to work effectively with others. This involves the skill to work together while respecting the needs of others, contributing, and making decisions. Collaboration is one of the essential and decisive establishments in the learning and work process. This was proven by a study conducted by Pifarré and Li [49] The study found that people with good collaboration skills enjoy better performance in school. Another study found that training students how to work together (e.g., plan, make decisions as a group, set goals, manage time, approve roles, and create a positive group environment) improved the effectiveness of collaborative learning. Through collaborating activities, students can solve problems given to them. Collaboration skills can be nurtured and developed, and the student's acquisition of these skills can be measured over time in the learning process. It can also be emphasized that collaboration skills can be cultivated and developed in the students through a proper learning process. With good collaboration skills, students can show good learning performance. Learning that involves cooperation among the students will produce a positive effect. That is, the students will be more comfortable sharing with their peers. Their peers will help, guide, and support each other to grow and learn through interaction and collaboration. Peer-involved teaching reduces stress through peer guidance, support, and feedback. Students will be able to build their self-confidence. Participating in learning with peers reduces anxiety and stress. Students may gain more confidence by receiving guidance, help, and peer feedback. Thus, implementing the HOTS-oriented E-PjBL tools allows students to establish good relationships with their peers, positively influencing the learning process. In addition to these three abilities developed in this learning model, another skill developed is communication skill. In learning with the HOT-oriented E-PjBL tools, the students can express ideas according to their points of view. In addition, at the step of presenting their self-created product, the students will develop their speaking abilities by orally presenting products using good pronunciation, intonation, and clear articulation. The students are more confident in expressing and communicating ideas in this learning process. By familiarizing the students with how to express ideas, they will develop communication skills. Communication skills are essential for students. Hence, the skills are basic requirements that everyone must master. Communication skills are a person's ability to convey or express their thoughts to others. Effective communication will greatly help children in the process of interpreting situations andcan improve relationships with others. Communication skills that develop in early life will help the children's social-emotional development. This means that the communication skills that the students should acquire must be good and effective. Good communication skills will allow the students to know and build good relationships with others and the outside world. Based on this explanation, it can be concluded that the implementation of the developed HOTS-oriented E-PjBL tools can effectively improve the 4C skills of elementary school students.
The finding of the effectiveness of the HOTS-oriented E-PjBL tools toward improving the 4C's skills can contribute to the science learning processes of elementary school students. Theoretically, conducting PjBL syntaxes by involving the related HOTS can train elementary school students to optimize the uses of 4C skills. Thus, the learning can be meaningful in improving their learning goal achievement and enhancing the4C skills of the elementary school student that are relevant to their ages and their phycological development level. Practically, the teaching that implements the learning tools will make the students learn to solve the problems given through completing a project. Through this learning activity, the students will develop 4C skills, and the learning tools are designed to create a more engaging learning atmosphere that motivates students to learn.
Second, there was a significant difference in science learning outcomes between students who learned with the HOTS-oriented E-PjBL and those who learned with a conventional tool. Learning with project-based tools in a HOTS-oriented network created a more enjoyable learning atmosphere for students. The developed tools had materials that followed the curriculum, applied attractive media design following the materials and the characteristics of elementary students, and had lesson plans as a guideline in conducting the learning activities. The tools made in compliance with the curriculum and learner characteristics will undoubtedly affect the attainment of the learning objectives. The tools developed following the principles of development will be able to be used appropriately. They can create interactive learning, and be inspiring, fun, and challenging; they can motivate the students to participate actively and affect their development, creativity, and independence according to their talents, interests, and physical and psychological development. In addition, the tools are an external factor in the students' learning process because it can strengthen their internal motivation. With the increase in the students' motivation in the learning process, the learning process will positively affect the attainment of the learning goals. The students with high motivation in learning will have a better learning experience, making their learning meaningful. With the learning experience that they received, they will be able to answer questions given to them. Thus, their learning outcome will improve. In addition, their learning process will focus on the problems they face every day, and the materials used are those that are real problems that they meet every day that they have to solve in the classroom. Students who are highly motivated to learn will achieve good learning outcomes. They will be more active in the learning process, which will undoubtedly affect the experience they get in the learning process that makes the learning process more meaningful. With the learning experience that they obtained; they will be able to answer questions given to them to get better learning outcomes. In addition, the learning process gives more emphasis on the problems that are faced in everyday life and the materials used are found in their environment [48]. This is because PjBL involves students in real problems seen every day, which they will solve in learning, where the resulting solutions are expressed in the form of projects. The learning focused on problems and projects will familiarize students with high-level thinking skills. PjBL learning effectively builds HOTS thinking skills [50]. Online PjBL is effective in the learning process. This is seen in the precise feedback and good interactions between learners and educators So, learning with project-based tools in a HOTS-oriented network can effectively enhance students' learning outcomes.
A learning outcome is the student's ability and skills after learning, which can make changes in their behaviours in terms of greater knowledge, understanding, attitude, and skills than before. The five types of learning outcomes are 1) intellectual skills which include learning concepts, principles, and problem-solving; 2) cognitive strategies, that is, the ability to solve problems; 3) verbal information skills, that is, the skill to describe something in words based on relevant information; 4) motor skill, that is, the skill to carry out and coordinate movements with muscles; and 5) attitude, that is an internal ability that influences a person's behaviour which is influenced by emotion, belief, and intellectual factors. Students could develop learning outcomes well through the PjBL learning process. The students carried out the activities by following the guidelines in the worksheet and with the help of the teaching materials that had been developed. According to these explanations, it can be said that project-based learning in a HOTS-oriented network allows students to learn more effectively. and increases their motivation in the learning process which will undoubtedly impact their learning outcomes. In this case, the increase in learning outcomes refers to the increase in the ability of the students to solve problems, their intellectual ability, which consists of the ability to learn concepts, and principles, solve problems, and describe things in words based on relevant information. These abilities were acquired because the tools that had been developed are compatible with the curriculum and the characteristics of students.
Third, simultaneously, there is a significant difference in 21st-century skills and science learning outcomes between the students who learned with the HOT-oriented online PjBL tools and those who learned with the conventional tools. As explained previously, it is related to improving 21st-century skills and science learning outcomes after the students were taught with the HOT-oriented online PjBL tools. In the learning process with the HOT-oriented online PjBL tools, the learning atmosphere created was more interesting, making the students more interested in education. Interest is the existence of curiosity or interest in an idea, thought, or event involving attention. Learning interests will increase if the learning tool, in this case, the learning material used, matches the students' interests and is interesting to learn. Someone interested in what they are learning will remember it easily in the long run and reuse it as a basis for future learning. Based on this explanation, it can be said that students who have an interest in education will be able to learn well. In addition, they will also be active in gathering information to find the right solution to a given problem and in solving the problems in the learning process, which will improve their 4C skills. The 4Cs skills are the skills needed by students in the 21st century. The priority is given to four skills: critical thinking skills, creative thinking skills, communication skills, and collaboration skills. The 21st-century skills must be taught at all levels of education, especially in elementary school. The students can use these four skills in learning to improve high-level thinking skills. The increase in the 21st-century skills in a sustainable manner will also increase learning outcomes as the increase in the students'4C skills will cause an increase in their learning outcomes. So, it can be said that the four skills, critical thinking skills, communication skills, and collaboration skills, will affect students' learning outcomes.
Based on this explanation, the advantages of the HOTS oriented online PjBL tools are as follows: 1) the HOTS oriented online PjBL tools developed are suitable with the current condition, where the learning process is carried out online; 2) the HOTS oriented online PjBL tools developed have materials that are compatible with 2013 the Curriculum 2013; 3) the HOTS oriented online PjBL tools that were developed have design and illustrations that are suitable with the materials and characteristics of the students and this made the students more interested in the learning process; 4) the HOTS oriented online PjBL tools developed could create a more relaxed learning atmosphere, both in terms of time and place where the learning takes place; 5) the HOTS oriented online PjBL tools developed undoubtedly make learning easier because it has been packaged in accordance with the conditions of students, parents, teachers, and schools; and 6) the HOTS-oriented E-PjBL tools developed contained learning centred on HOTS activities that will undoubtedly influence the 4C skills and student learning outcomes.

V. CONCLUSION
The developed HOTS-oriented E-PjBL tools can effectively enhance 4C skills and science learning outcomes of elementary school students. This can be seen from the difference in mean scores, where the mean for the experimental group was better. Another finding was that the HOTS-oriented E-PjBL tools affect21 st -century skills and science learning skills. The effectiveness of the developed tools cannot be separated from their characteristics. The learning process was carried out online, used the materials that fit well with the curriculum currently used, or the 2013 curriculum, and had a design and illustrations that matched the materials and characteristics of the students who used them. The students were more active participants in the learning process. The tools have created a more relaxed learning atmosphere both in terms of time and place where the learning took place and made learning easier for the students because it had been packaged following the conditions of students, parents, teachers, and schools. Thus, learning with the HOTS-oriented E-PjBL tools can be recommended to improve the 4C skills and science learning outcomes. The limitations of this study can be seen from the limited number of test subjects. Further research can test it on subjects at different levels and in larger numbers.

CONFLICT OF INTEREST
The authors declare that there is no conflict of interest regarding the publication of this manuscript. In addition, the ethical issues, including plagiarism, informed consent, misconduct, data fabrication and/or falsification, double publication and/or submission, and redundancies have been completely observed by the authors.

AUTHOR CONTRIBUTIONS
Ni Wayan Rati contributes in conceptualization, methodology, original writing draft preparation, writing responds of reviewers and editing text of the original paper; Ida Bagus Putu Arnyana gives supervision in conceptualization and validation of data; Nyoman Dantes and Gede Rasben Dantes contribute in supervision of methodology, statistics and validation of data. All authors have read and agreed to the published version of the manuscript.