Machine Vision Algorithm Training Course Construction with PBL

—Interdisciplinary integration of theory and practice is imperative as a course requirement in emerging engineering education, and in the public elective course "Machine Vision Algorithm Training". Considering the entire teaching process, including pre-training, in-training, and post-training, this paper discusses the course construction and content in detail in terms of project-based learning (PBL). The PBL teaching approach and evaluation methods are described in detail through a comprehensive face recognition training case based on a convolutional neural network (CNN) and Raspberry Pi. Through project design training from shallower to deeper, interdisciplinary integration of theory and practice is cultivated, stimulating interest in course study. The results demonstrate that PBL teaching improves the engineering application and innovative abilities of students.


I. INTRODUCTION
Since February 2017, the Ministry of Education of China has actively promoted development of emerging engineering education [1]. Unlike traditional engineering majors, new engineering majors focus mainly on internet and industrial intelligence, and cultivate innovative and diversified engineering skills. Machine vision is an important branch of artificial intelligence (AI). A convolutional neural network (CNN) is a typical deep-learning technology applied in machine vision [2]. CNNs have been widely used for image recognition and object detection [3], [4]. In references [5], [6], deep learning and Raspberry Pi were applied to the internet of things (IoT). Experiment teaching effectively unifies theory and practice. Exploratory experiment teaching effectively fused research into course teaching in reference. Practical machine vision projects can stimulate student interest in learning and broaden application of deep-learning in computer vision [7].
The project-based learning (PBL) approach is an active, student-centered, and problem-centered small-group method that encourages independent thinking and creativity. Manuscript  Reference [8] shared the learning ecosystem of a projectbased embedded systems course, identifying course elements that support self-directed learning and how assignments guide students toward becoming adaptive experts. In references [9], it introduced PBL features for developing technological, curricular, and pedagogical supports to engage students in computational thinking through modeling. PBL uses professional knowledge in the context of problem-solving to develop innovation and creative thinking [10]. Reference [11] discussed PBL approach for improving the engineering application and innovative abilities of students. PBL teaching approach has been widely used in many courses and obtained significant improvement of teaching effects [12], [13].
With promotion of the "Emerging Engineering Education" construction plan, considerable research has been conducted on teaching methods. To improve interdisciplinary applications, enhance the fusion of theory and practice, and enrich the innovation and entrepreneurship education curriculum [14], the Innovation and Entrepreneurship Center of East China University of Science and Technology (IEC-ECUST) began offering an interdisciplinary and interprofessional public elective course titled "Machine Vision Algorithms Training" from January 2021. The course is offered to all students and is not dependent on their majors. Focusing on the cultivation goals of applied and innovative talents, guided by the PBL approach, this paper discusses the course construction and teaching implementation. Taking the exploratory training project of face recognition combining software and hardware collaboration as an example, this paper demonstrates the detailed teaching process based on PBL. Finally, it analyzes the assessment methods and teaching effects. The results demonstrate that through a multi-dimensional linkage model of "brain, hands, and mouth", interdisciplinary integration of theory and practice is cultivated, improving the ability to engineer and innovate, further stimulating interest in course.

II. COURSE CONSTRUCTION WITH PBL
A. Course Construction Background "Machine Vision Algorithms Training" is an interdisciplinary public elective course. Most of the students in the class have a strong interest in the field of artificial intelligence, and intend to use artificial intelligence technologies to solve some engineering problems in their majors through theoretical and practical learning, and can deeply participate in innovative activities or competitions. In major events such as innovation and entrepreneurship Machine Vision Algorithm Training Course Construction with PBL Cuiling Jiang, Yongjing Wan, Yu Zhu, and Rong Wang training programs for college students, "China Internet+" contests, and the "Challenge Cup" competitions, most student teams are interprofessional, and the members in the team need strong and comprehensive software and hardware design and development capabilities. Although some existing public elective courses can provide certain support for student basic ability and professional ability training, it is necessary to provide a comprehensive course including design and training, effectively integrating the skills in the previous courses, and aiming to improve student software and hardware collaborative design and development abilities. So, "Machine Vision Algorithms Training " course is offered.
Through systematic learning and training in the course, it cultivates student engineering application ability and innovation ability to solve complex engineering problems. It also lays a solid foundation for subsequent participations in many innovation and entrepreneurship projects and various kinds of competitions.

B. Course Construction System with PBL
PBL uses professional theoretical knowledge and practice, with the specific teaching aim of designing products that are closely related to engineering practice. Students work in groups to formulate project plans, and implement them through discussion and communication. In completing projects, students learn and apply new knowledge through "learning by doing" to improve engineering application and innovation abilities. PBL is student-centered, encouraging enthusiasm and initiative in learning, and cultivating engineering design and scientific thinking ability [8], [9]. The PBL educational philosophy runs through the entire course teaching process, including content teaching, project training, and extended application. The PBL curriculum construction system is shown in Fig. 1.
With the rapid development of internet and information technologies, many kinds of teaching modes have been introduced in class. The traditional teaching mode is transformed from face-to-face teaching to online teaching in times [15]. Due to the COVID-19 pandemic, face-to-face courses in most of educational institutes have changed to online courses [16]. Teaching modes such as online-offline mixture, SPOC, or MOOC are widely conducted in courses for improving teaching effects.
In the teaching process of "Machine Vision Algorithms Training" course, online resources are used, for a mixture of online and offline teaching. Offline teaching is conducted in the computer room; students can use the Chaoxing learning platform on a mobile phone or an online computer. Training project design progresses from shallower to deeper. PBL includes three sections: pre-training, in-training, and post-training, and cultivates individualized and diversified student abilities. Achievement of course objectives is evaluated, producing a closed loop that promotes continuous improvement in teaching.
In pre-training, teachers use the Chaoxing platform to present the training project content and requirements. Teachers provide project videos or learning materials related to the classroom content that students can preview ahead of time. For example, when teaching the fundamental methods of filtering and threshold processing, a training project titled "Beautiful photo show" is provided, with life photos of interest to students. Such projects stimulate student enthusiasm for learning. Students preview, consult materials, discuss, and propose a plan before class, greatly improving in-class project training, as shown in Fig. 1.
"In-training" is conducted simultaneously using the offline computer room and the online Chaoxing platform. Teachers prepare 2-3 questions in advance as exercises or for small discussions in class. The selected topics are as short and refined as possible, and are closely related to the current teaching. The allotted time is generally 3-5 minutes. With online learning statistics, teachers can monitor the accuracy and completion of student work, and provide guidance for common problems or weak areas.
"Post-training" is conducted online and offline. Exploratory training projects with comprehensive designs are conducted using a combination of software and hardware to stimulate independent student inquiry and improve complex engineering problem-solving ability. For current popular deep-learning technologies, comprehensive training projects that include software and hardware collaboration are presented. Teachers present the functional requirements and design ideas. Students can review materials, discuss in groups, or consult teachers to complete the project design and testing. The teacher scores and evaluates the completed project; students also evaluate each other.

C. Training Content Design
In PBL, each training project improves theoretical understanding. Through the project design, code writing, and case training, students analyze and solve specific engineering problems related to machine vision. Comprehensive abilities in design and software and hardware collaborative development are improved. The project training is divided into basic content, design content, and exploration content for advancement of engineering and innovation abilities. The PBL training content is shown in Table I. With the rapid development of deep learning, exploratory project design focuses on deep-learning applications. Deep-learning models are trained and tested on a computer using a CNN such as Unet. The exploratory projects include flower recognition, Unet medical image segmentation, face recognition, and other deep learning applications. The trained models are imported into Raspberry Pi to control a manipulator or smart car for comprehensive application combining software and hardware.

III. EXPLORATORY TRAINING PROJECT BASED ON DEEP LEARNING AND RASPBERRY PI
A comprehensive training project was conducted in Tensorflow2.4 and Python3.7 environments using Raspberry Pi 4B. Raspberry Pi is an ARM-based microcomputer mainboard with an HDMI interface for an external keyboard, USB, mouse, and display, and an SD/MicroSD card for storage. Through the access switch of face recognition control based on a CNN and Raspberry Pi, project design and collaborative development are undertaken through PBL. A CNN network is built on a computer and trained using a face dataset. The trained CNN model is imported into Raspberry Pi, which uses the CNN to detect and recognize faces in real time, and controls the opening and closing of the access door using the steering gear. The detailed training process is described as follows.

A. Task Release
The teacher publishes the training project requirements on the Chaoxing platform. Students freely form groups of 1-3 people, and produce an overall functional block diagram according to the training requirements and learning materials. The overall functional guideline is shown in Fig. 2; it contains software and hardware. The software part includes CNN model construction, training, and testing. The hardware part includes the Raspberry Pi for face recognition and control of the access switch.

1) Software design
This includes CNN network design, model training, and data testing. The face recognition functional block diagram is shown in Fig. 3. In the CNN model, students can choose the number of convolutional layers, the Softmax classification function, and the activation function. In training, a low recognition accuracy from underfitting may result from an insufficient number of network layers. Students can discuss in groups, access information online, or communicate with teachers to optimize the model and adjust parameters to improve recognition accuracy. The network structure of this experiment suggests three convolutional layers, three pooling layers, and two fully connected layers. Students can adjust the number of network layers to meet the Raspberry Pi hardware requirements. In training, CNN face recognition with an accuracy rate exceeding 90% is considered an optimal solution. The training dataset contains six face types, with a total of 630 pieces. The dataset is split in a 9:1 ratio; 90% of the data is used for training and 10% is used for testing. After CNN model training is completed, students test the model using the testing dataset to verify the model performance. The test results are shown in Fig. 4, which indicates that the trained CNN model can accurately recognize a variety of faces. After completing CNN model training and testing on the computer, students save the model and import it into the Raspberry Pi.

2) Hardware design
The hardware functions are based on the Raspberry Pi 4B, which has an HDMI interface for an external keyboard, USB, mouse, and display, and an SD/MicroSD memory card. Students import the saved CNN model into the Raspberry Pi. The Raspberry Pi camera collects the video stream and transmits it to the Raspberry Pi. The Raspberry Pi recognizes faces using the CNN, and controls the steering gear through the PWM interface to access the face. The functional block diagram of the hardware is shown in Fig. 5. Students determine the hardware design plan according to the learning materials provided by the teacher on the Chaoxing platform or online, and can discuss with the group and the teacher. In project training, teamwork and communication are important. Problems are carefully explored and solved.

C. Project Testing
To further verify the comprehensive design and implementation abilities of the students in using deep learning and Raspberry Pi, the teacher grades and evaluates on the spot. The students also evaluate each other and other groups. Some students arrive at the site early for debugging and drills. The enthusiasm for learning is high, and is further motivated by competition between groups. Fig. 6 shows the student on-site training. The quality statistics for the 28 groups (59 students) that participated in the exploratory training in the fall 2021 semester are shown in Table II; 82.2% of the groups mastered and applied the professional knowledge they acquired to better complete the comprehensive training task. Students who produce high-quality results can serve as assistant teachers to share their experience with new students; their results are submitted to the Chaoxing platform as an example. Teachers follow up in time with students who were less successful; usually, they did not participate in class, or had limited professional knowledge and difficulty learning.
Teachers must help students set learning goals. Teaching feedback provides a powerful means for further improvement in teaching.

IV. ASSESSMENT AND IMPLEMENTATION
A comprehensive assessment of the course focuses on individualized training and process assessment. Teachers track student learning and trajectory during the course, including online learning on the Chaoxing platform, completion of classroom exercises, discussions, homework, completion of training projects, and comprehensive on-site experimental evaluation. The Chaoxing platform provides detailed process records; the course assessment content is shown in Table III. This course was offered in the spring and fall of 2021. Academic data analysis from the two semesters is shown in Table IV. The number of students in the fall of 2021 increased significantly from the number in the spring of 2021. The average course grades and overall satisfaction with the course significantly improved. This training course using PBL was generally appreciated by the students, and produced the expected learning. In Fig. 7, the proportion of good grades in the fall of 2021 was greatly increased, indicating that most students improved their professional abilities through the course.

V. CONCLUSION
The training course has greatly benefitted students. For the students in the School of Information Science and Technology of ECUST, interest in the field of machine vision was enhanced, and comprehensive design capabilities were improved by the training. Some students won awards, including the National Silver Award in the 2021 "China Internet +" competition, the first prize in the 17th Shanghai "Challenge Cup" in 2021, the second and third prizes in the 2021 Shanghai College Students Computer Competition, and the second prize in the 23rd China Robotics and Artificial Intelligence Competition (Shanghai Division). Winning these awards has further stimulated student interest and improved multi-disciplinary cross-integration application and innovation capabilities.
The "Machine Vision Algorithm Training" course was considered in this paper in terms of course construction and training content using PBL in the context of emerging engineering. In the training, student engineering application and innovation abilities were greatly improved. The course teaching provides a strong basis for further exploration of innovative education measures that integrate production.

CONFLICT OF INTEREST
The authors declare no conflict of interest.