Attempts at Learning Creative Problem-Solving in Remote Schools : Professional CPS Student Evaluation Using a Video Calling Application

Abstract—According to the elementary and junior high school curriculum guidelines of Ministry of Education, “Problem finding and solving ability” is listed as one of the qualities and abilities that are the basis of learning. However there are some challenge for teacher to evaluate students. So, we hypothesized “the evaluation by the professional” would be effective for evaluation. In this study, evaluations by corporate experts, given via video calls on a tablet, were used to help improve students’ creativity and problem solving as part of CPS learning.


I. INTRODUCTION
According to the Ministry of Education's elementary and junior high school curriculum guidelines (2017) [1], "problem finding and solving ability" is one of the qualities and abilities basic to learning. Yumino et al. (2006) [2] describe that problem solving can be categorized as either typical or creative. Typical problem solving is a learning method in which problems be solved immediately with the knowledge and skills learned at school. On the other hand, creative problem solving does not lead to a good solution unless the students add new knowledge to their previous learning and experiences. Kawakita (1993) [3] states that creation is "the ability to solve problems, not the ability to invent and discover," and creativity is "to break through the current situation and constantly change it into a new state." Nishiura (2011) [4] presents three creative actions that embody creativity: spontaneity, lack of model, and compellingness.

A. The Problem
Although many of the current educational practices engage students in problem finding and solving, the methods used to evaluate student learning remain a challenge. This is because project-type learning such as the kind used for developing problem finding and solving ability is difficult to evaluate using conventional paper tests. Manuscript  There are three general evaluation methods: evaluation by teachers and evaluation by learners (mutual evaluation or self-evaluation). Kage (2004) [5] described that "It is effective to conduct evaluation within the learning community between the teacher and the learner in order to evaluate interest, motivation, and attitude in the new concept of scholastic ability." This points to the benefit of such evaluations. And, there are many studies on evaluation by learners. For example, practice based on presentation skills (Okura et al.2004) [6], practice of communication in foreign language education (Okura Murao, 2003) [7] and practice in project learning (Matsumoto, 2000) [8].
However, despite the effectiveness of methods that incorporate self-assessment and mutual assessment, challenges remain. As Fujiwara et al. (2007) [9] point out, the problem in self-assessment is that some learners do not reflect on their faults. In mutual evaluation, learners want to be highly evaluated and a higher evaluation may be given to expectations.
For these reasons, recent practical studies have incorporated both self-evaluation and mutual evaluation. Kaneda (2019) [10] studied problem solving and concluded that "By performing mutual evaluation, children have a new perspective and can improve their lives by further improving their tasks." On the other hand, Kaneda also found that "only a few of the children pointed out improvements in practice," and they could not think seriously about the challenges of others. Thus, it is difficult from the viewpoint of equality and reliability to evaluate problem solving learning products by relying only on the students themselves.

B. Study Aim
As a possible solution, this study examines the effectiveness of "expert evaluations" of creative problem solving (CPS) learning products, given to students via video calling.
Students at schools in remote mountainous areas took part in CPS learning in three classes. We matched a company located in the city with each class theme and asked the company professionals to evaluate the students' learning products (ideas). Since it is difficult for remote schools to request field trips to companies in the city, and it is likewise difficult for companies to sponsor such trips, video calling was used to facilitate the expert-student interactions. The students' creativity was measured before and after the classes.

A. Development of CPS Materials
We developed new CPS learning materials for the study. To promote spontaneity in the classes, themes were selected that would be of interest to the students. The author interviewed students at each school to identify which subjects they liked. Accordingly, teaching materials were developed for the following three themes: automobiles, earthquake and disaster prevention education, and sports.

B. Participants
A total of 113 students in grades 4 to 7 participated. In addition, corporate experts participated.

C. Classes
The classes were set up as shown in Table I. A tablet and the free message application "LINE" were used to make the video calls. Each class met for three sessions that incorporated CPS learning. The CPS learning process consisted of six steps, following Nishiura (2011): (1) confirmation of the learning objectives, (2) collection of goal lists, (3) clarification of the problems, (4) idea generation, (5) idea selection and reinforcement, and (6) action planning. A class session (45 minutes * 3 frames) was organized with reference to the drafting. In the first and second sessions, students were divided into groups of four to five, and they tackled the CPS issues given by the corporate experts. In the third session, the students presented their work products (ideas) to the corporate experts. The experts evaluated the students' ideas in terms of the time they had spent developing them, willingness to solve problems, and novelty of the deliverables (ideas). To illustrate, Table II shows the flow of the class on automobiles.
In the automobile class, students were challenged to imagine a "future automobile" designed with the environment, safety, and convenience in mind. In the earthquake and disaster prevention education class, they were asked to think about a fire emergency at school and brainstorm ways to quickly and accurately convey information about the fire as well as execute an evacuation route. Finally, in the sports class, the students discussed what type of "sportswear" and "goods" would be appropriate for large sports events. ・Learn about current automobiles from the three perspectives of environment, safety, and convenience. ・Study past issues and solutions from the same three perspectives.
①Confirmation of purpose ② Clarification of the target list 2 ・Consider automobile issues from the perspective of the environment, safety, and convenience. ・Identify one problem and think of a solution (brainstorming). ・Discuss the ideas in a group and write down a specific plan.
③ Clarification of the problem ④ Idea generation ⑤ Selection and reinforcement of ideas 3 ・Collect ideas on worksheets and prepare to give a presentation. ・Use video calling to present the results to automobile developers in the city center. ・Listen to the corporate experts' evaluations, approval, and ideas for improvements of the ideas. ・Think of ways to improve the ideas based on the experts' comments. ・Review the learning experience and fill out the questionnaire.
・Evaluation by experts ・ Consideration of improvement measures ・Self-assessment for which the answer is fixed"

＊ I like to try new things
To measure the effect of the practice, we administered a questionnaire survey about students' creativity before and after the class. For each questionnaire item, the answer choices were "very much," "quite a lot," "a little," or "not at all," scored from 4 points to 1 point, respectively. A t-test with a corresponding significance level of 5% was conducted. Tables III to V show the results for each class.  In addition, in order to consider the students' thinking with respect to the theme of each class, a mind map survey was administered before and after the class. The contents of the students' free descriptions were classified and analyzed qualitatively. Fig. 3 show the results for each class. In the automobile class, there was a significant difference of 5% with respect to the average value of "thinking about what people do not often consider." The average value for this item increased from 2.625 before the class to 3.042 afterwards.

Fig. 1 to
As Fig. 1 shows, the number of words related to the environment and safety increased, suggesting that the students' image of automobiles changed. The terms "face recognition" and "speak" can be seen as "convenient functions other than "run,"" it can be said that the students' ability to create new things was cultivated in the classes. In the earthquake and disaster prevention education class, there was a significant difference of 5% with respect to the average difference in awareness of "take on the challenge of new things." Specifically, it increased from 2.750 to 3.042 after the class.
As Fig. 2 shows, traditionally, disaster prevention education in Japan has consisted only of annual evacuation drills with few attempts at other types of learning. In the free descriptions, words related to "disaster prevention equipment" increased, and there was an awareness not only of evacuation drills carried out at school but also environmental aspects such as equipment and surrounding places. In addition, the children said that it was important to always think about the fact that the corporate experts believed they were all right (normality bias). In other words, there was an opportunity to think about disaster prevention from a multifaceted perspective, not just conventional disaster prevention drills and evacuation drills. In the sports class, there was a significant difference in the average difference in the awareness of "taking new things" and "thinking about what people do not consider often" at a significance level of 1%, as shown by the increases from 3.138 to 3.369 and from 2.462 to 2.738.
According to Fig. 3, the phrase "work" includes not only professional athletes, but also descriptions of occupations such as product development and planning, which led to the development of a broader view of the sports industry. Negative images such as "get tired" and "hot" emerged before the class, but afterwards, words such as "enjoy" and "cool" were more frequent. The classes placed importance on the various ways of engaging with "play, see, support, and know" sports, and the children's interest in "support" sports increased. Table VI shows the questionnaire results for each class (Tables III to V). Fig. 4 presents the results of the free descriptions. In the total questionnaire results, there was a significant difference at the 1% significance level with respect to the average difference in awareness of "taking on the challenge of new things" and "thinking about what people do not consider often." The numbers increased from 3.133 before the class to 3.336 afterwards and from 2.469 to 2.761, respectively.
Even in the free descriptions, the total number of words increased by 72.5%, suggesting that the students improved in imagination and creativity in each class.
There was a significant difference in the items related to 1) motivation for creativity (sustainable motivation) and 2) novelty (creating new ideas and new ways). Specifically, students' motivation increased.

IV. DISCUSSION AND CONCLUSIONS
In this study, evaluations by corporate experts, given via video calls on a tablet, were used to help improve students' creativity and problem solving as part of CPS learning. The findings show that the learning objectives were largely achieved.
Furthermore, it was effective to carry out the classes using video calls. This means that schools in mountainous areas in Japan, where there is less diversity and fewer educational facilities than in the city center, can implement video calling for student learning. This may help reduce the educational disparity of career education due to geographical restrictions.
However, although the workload on corporate partners is light, implementing the curriculum at all schools may not be realistic due to the requirement of corporate cooperation. Also, when learning incorporates evaluation by corporate experts, it is necessary to encourage the students to acquire the skills to convey their interests and ideas.