Experimental Evaluation of OOP Concepts Learning with a Kit-Build Concept Map: A Case Study

 Abstract —A kit-build is a framework that utilizes a concept map to visualize knowledge and automatically assess the concept map that was created by the learner by recomposing the kit. In this case study, a kit-build will be implemented in object-oriented programming learning to investigate its usefulness in increasing learning gain compared with a traditional summarizing activity. In this study, learners will practically use a kit-build concept map system to facilitate reviewing and reconstruct their knowledge of object-oriented programming learning resources. This study conducted an experiment using two groups: the summarizing group (n = 46) and the kit-build group (n = 35). The variables that were measured were the pre-and post-test scores and learning gain. To evaluate the significance of the difference between the two groups, a non-parametric ANOVA analysis was performed. The experiment result confirms that incorporating the kit-build concept map as a knowledge visualization technique could improve not only the learning performance but also the overall learning gain compared with the group that used the regular summarizing method.


I. INTRODUCTION
One important skill when studying computer science is Object-Oriented Programming (OOP).Most universities' computer science departments include OOP in their curriculum, and the students must take it to successfully pass the course.The basis of OOP learning is translating real-world objects into Object-Oriented (OO) code.It is important to understand OOP concepts to be successful in the domain of learning OOP [1,2].Learning OOP is considered a difficult task in terms of understanding the fundamental concept of objects [3].There is a report on the difficulties of learning complex objects for novice programmers and learners [4] in terms of identifying objects and creating OO designs.A report by [5] concluded that novice programmers and learners are limited to surface and organized knowledge that is based on superficial similarities.
Numerous tools and methodologies have been developed to help learners to understand the concept of OOP [6,7].A report on research about the development of an educational environment to help students to understand and master the basic concepts of OOP concluded that the environment and teacher supervision in programming activities could help students to solve programming problems [8].Another report noted that teaching OOP with games provides a positive and supportive atmosphere in which learners can learn the principles of OOP [9].Most of the tools and methods that have been introduced directly introduce learners to the programming implementation of OOP without contextually explaining the concept of OOP [1].It is important to provide an excellent learning experience by conceptualizing OOP's basic features and concepts before moving toward the technical details of the programming language [1].
Learning concepts by reading a textbook or through other reading materials could be challenging because the words that are written and combined into meaningful ideas might not accurately portray the meaning and ideas of the concept [10].Summarization is often used as an aid to help learners in recalling and reconstructing their knowledge after reading.A popular strategy to aid the students' understanding of concepts in the reading materials is a graphical and visual strategy.Concept mapping is a strategy that helps learners to organize and structure complex information through visual aids [11] and improves their critical thinking [12] based on the principle of a concept map, which is a diagram that shows the relationships among concepts.
A Kit-Build (KB) concept map is a concept mapping activity framework [13,14].KB has been proven to be similar in terms of efficiency to the normal concept mapping method [15].In the KB activity, learners are provided with concept map components called -kit‖ and are supposed to rebuild the kit into a complete map called a -learner map.‖The kit was created by decomposing the actual completed map, which is called -goal map‖ and is created by the teacher before the KB activity is performed.
The KB approach enables teachers to share their knowledge with learners and promote collaborative learning between the teacher and learners [16,17].This collaboration enables learners to visualize their knowledge using -kit‖ as a form of the teacher's knowledge and the teacher to point out where the learner is struggling to give more direct feedback.The KB framework also assesses the similarity between the learner map and goal map, which is called the -map score,‖ so the teacher can investigate the learner's level of understanding.
By incorporating KB as a concept mapping activity to achieve knowledge visualization, this study aims to improve learners' learning gain in the domain of conceptual OOP basic features and concepts.By using a KB concept map as computerized concept mapping, the research questions will be as follows: Can using a KB concept map as knowledge visualization media increase learning in learners compared with the traditional summarization method (RQ1)?What is the correlation between the learners' learning gain and the KB map score (RQ2)?

II. BACKGROUND A. Previous Research
There is a previous research on learning OOP.It is concluded that the OO concept must be understood to ensure success in the domain of learning OOP [2].The integration of knowledge, skill, and strategies can serve as a framework to support novices learning OOP.OOP is a different paradigm compared with procedural programming, and it is reported that transitioning the paradigm is challenging for learners [18].
In the past decade, several studies have been performed to enhance OOP education.An educational programming environment called JavelinaCode has been used to enhance learners' OOP skills [19].The report concludes that the comprehension of OO concepts, such as inheritance, polymorphism, and OO design, is an important factor in learning the OOP.There are numerous tools available to aid OO learning, such as Jeliot [20] and Alice [21] which are used as a learning environment in which learners can learn to use code to define objects.Greenfoot [22] is an educational software creation platform that uses OOP learning scenarios to understand the technical specifics of the code [1].
Much research to enhance OO learning focuses on teaching learners the technical details of programming rather than the basic knowledge and concept of OOP.One report concluded that in the beginning of the study, learners had difficulty in solving complex problems of OOP, as they needed to understand basic concepts and knowledge to build a higher cognitive understanding of advanced programming concepts [23].Supportive systems, such as an online environment [24] and virtual classroom [25] have been used to aid OO learning; however, they still lack the specific delivery of basic knowledge and concepts of the OOP paradigm.

B. OOP Learning Strategy
One OOP learning strategy is understanding the basic concepts of OOP, which are classes and objects; encapsulation; inheritance; and polymorphism [26].OOP concepts can be learned by reading textbooks or using simulation tools.The learning outcomes according to Thota and Whitfield [27] can be divided into four stages: essential facts and concepts of OOP; theory and implementing the program; use of OO design and software modularity [28]; and organization and internalization of values.In this study, the KB approach focuses on the first stage-the essential facts and concepts of OOP.This stage is divided into four sub-stages: recognize-base knowledge, domain vocabulary; trace-desk check a coding solution; interpret-code a low-level solution; and translate-interpret and convert code.KB concept mapping is considered in the domain of learning media that supports the first stage of essential facts and concepts of OOP: -recognize-base knowledge, domain vocabulary.‖This is the stage in which learners learn and demonstrate their knowledge and understanding of essential facts and concepts relating to OOP [29].Fig. 1 shows the domain of learning media in the first stage of OOP learning.Learning media is a means to convey information from teacher to learner during the learning activity [30].Learning media could be in the form of a printed or online copy (computer-supported media).In computer science education, computer-supported learning media for teaching and learning is a necessity [29].In this study, KB provides computer-supported learning media that could realize knowledge visualization in the stage of understanding the essential facts and concepts of OOP.
According to Thota and Whitfield [29], the learning media can be divided into five categories, which are as follows: narrative media-non interactive presentation; interactive-user controlled; adaptive-respond to user; communicative-informing, discussing; and productive-create models, descriptions.Existing tools that aim to aid OOP learning fall into the interactive and adaptive category.In the interactive category, the tools are provided in terms of program visualization and knowledge visualization.
There are existing tools in the program visualization category, such as Jeliot [20].In the adaptive category, the tools are provided in terms of an integrated development environment.There are existing tools in this category, such as BlueJ [31], Violet [32], and Junit [28].KB, using a concept map as an approach [13], is categorized as a knowledge visualization tool, as shown in Fig. 2.

C. Concept Mapping
Concept mapping is a widely used technique to represent knowledge in a graphical format [33].Concept mapping has been used for an educational purpose, such as assessing learners' knowledge, sharing meanings between people, planning the problem-solving process, organizing knowledge, and representing the learners' knowledge structure.In learning OOP, concept mapping has been applied to present the learner's knowledge structure and organize learning products in an OO Java programming course [34].The study International Journal of Information and Education Technology, Vol. 13, No. 9, September 2023 on concept map utilization has the learner presenting their learning status for every programming concept in the course [34].Experimental results show that the exercise completion rate for the learners who used concept mapping was significantly higher than regular learners who learned sing the regular method without using the system.
The concept map has been used to represent partial OOP knowledge that consists of prepositions that are decomposed and then structured to form the map.Concept mapping is used to direct learners to the topic content based on the result of the learner assessment of OOP learning [35].In the study conducted by Dogan et al. [35], the concept map was partially used in a system called Object Oriented Programming Tutor using Concept Map Model (OPCOMITS) to regulate the topic hierarchy to measure the learners' knowledge about the learning topic and stimulate their learning.Experimental results show that the difference between the pre-test and post-test scores was greater for the experimental group than the control group.This indicates that learners who learned using OPCOMITS with the addition of a concept mapping activity performed significantly better than learners who learned with the traditional system.

D. Kit-Build Concept Map
A KB concept map is a concept mapping framework that allows for the automatic diagnosis of a learner-created concept map [13].There is an activity in this framework in which the learner is tasked with reconstructing the concept map components called -kit,‖ which consists of nodes and links, into a complete map.The kit was provided by a teacher or expert and was derived from the course materials.The KB framework is a closed-end concept map, which means that instead of creating their own concept map from scratch, learners recompose the map using the provided -kit.‖The closed-end approach enables an automatic diagnosis of the concept map recomposed by the learner and a comparison with the actual map that was created by experts.
Because the learner is not developing the map from scratch, but rather using the kit provided by the system, the use of concept mapping may direct them to focus on the right approach of learning the information.Funaoi et al. [36] conducted a comparison of scratch-build and KB concept mapping.The results demonstrate that the KB group had superior memory retention compared with the Scratch Build (SB) group.Memory decline was also lower in the KB group compared with the SB group.This suggests that the KB method recommends memory retention during the learning process.
There are two tasks in the KB concept map strategy: segmentation and organization [37].Segmentation involves extracting the nodes and links (kit) of a concept map from the learning resources.The structuring task involves reconnecting the extracted pieces (kit) and recreating the entire concept map.Fig. 3 shows how the kit was extracted from the learning materials or resources into the -kit‖ and its form, which comprises concepts and links, in the segmentation task and how the kit was linked into the concept map in the structuring activity.
The KB framework includes three phases: goal map construction, learner map construction, and analyzer [38].
The goal map is created by an expert or a teacher using map components gathered from educational resources (segmentation and structuring tasks).The goal map is then broken into a -kit‖ comprising discrete concepts and links.The kit is subsequently distributed to learners so that they can reconnect to the learner map (structuring task).After learners have finished their task of creating the learner map, the system will automatically analyze and compare the learner map to the target map.Fig. 4 shows the KB concept map system's lifecycle flow.

E. Learning Gain
The normalized gain is used as a rudimentary assessment of a course's success in improving conceptual comprehension to quantify the learning gain [39].This metric is generally defined as -the quantity of knowledge learners gained divided by the amount of knowledge they could have gained.‖Normalized gain is a typical method for assessing the effectiveness of education [40].According to the study, normalized gain has a high association with learner abilities; thus, this measurement may be used to compare the impact of learning methods between traditional learning methods and KB idea mapping methods.The following is the normalized gain Eq. ( 1): International Journal of Information and Education Technology, Vol. 13, No. 9, September 2023 NLG is the normalized learning gain, or g-factor, in Eq. ( 1).PreScore is the pre-test score, while PostScore is the post-test score.The post-and pre-test scores are expected to be in a range of 0 to 100.If not, a scale-normalized score is required.The formula produced the ratios of the realized learning improvement to the maximum feasible improvement [41].The normalized gain measurement offers a considerable advantage over the traditional gain calculation, in which gain = post − pre.This allows teachers to compare gains in scores for many learner populations, irrespective of the pre-test scores.

A. Participants
In this study, the subjects of the experiment were 81 second-year undergraduate learners from the Information Technology Department of State Polytechnic of Malang, Indonesia.All the learners had no knowledge or prior experience of building concept maps.The learners were divided into two groups: 35 in the KB group and 46 in the control group.The divisions between the two groups were based on the course classes that were randomly distributed at the beginning of the semester.Learners who did not attend the class, did not complete all the experiment activities due to system error, or left blank answers to the tests were not included.In the KB group, learners used the KB system after the teacher had delivered the learning material presentation.In the control group, learners used the summarization method after the teacher had delivered the learning material presentation.

B. Instruments
The instruments of this experiment were the learning material from the course module of the OOP course that was released by the teaching team at the Information Technology Department of State Polytechnic of Malang, Indonesia.The OOP concept that was used in this experiment was encapsulation.
The KB goal map that was used for the experiment consisted of 15 propositions.The tests that were used in this experiment consisted of 20 multiple answer questions.The KB goal map and test questions were created and verified by three teachers that taught and were in charge of the OOP course in the department.

C. Procedures
The experiments were performed over a total of 50 minutes.At the beginning of the experiment, both groups were given a 10-minute pre-test questions to measure the baseline of the learners' knowledge.The learners received learning material in the form of a teacher presentation that was 15 minutes long.The next stepmouss was the activity that differed between the KB group and control group.The KB group performed KB concept mapping using the KB system, whereas the control group performed summarization.Summarization is a common and acceptable method of learning in which the learner recalls what they learned from the teacher to retain the knowledge.In the summarization method, the learner summarizes the teacher's explanation.The content of the summary is up to the learner and is based on their understanding of the learning material explained by the teacher.
In the KB system, the learners were tasked to re-construct the kit into a learner map.To maintain a fair condition, the control group performed summarization on the keywords that were provided based on concepts and links that were similar to the one in the KB system.This activity was performed in 15 minutes for each group.After the activity, a post-test was performed, which took 10 minutes.This test contained the same questions as the pre-test.During the pre-and post-tests, all the materials were closed.Table I

D. Analysis
In previous research [38] the result of the learner performance test was analyzed on the post-test and delayed test.It showed that the usage of KB could increase learner performance compared with the traditional control group.Previous research on the effectiveness of concept mapping between concept map interventions and the conventional method [42] shows that concept mapping could increase learners' skills by investigating the score after concept map interventions (post-test score).In a previous study on enhancing learner high-order thinking (HOT), concept mapping was proven to enhance HOT in a laboratory educational activity by observing the pre-and post-test results [43].
In this study, three variables were used to perform the analysis: pre-test, post-test, and learning gain.The pre-test score was used for the baseline of the learner's knowledge of the learning material before the learning activity.The homogeneity test using the Levene test of the pre-test score was performed to ensure that both groups had the same level of OOP knowledge.The post-test was conducted to measure the level of knowledge after the learning activity.The learning gain was calculated and used to measure the effectiveness of the learning activity.
A non-parametric ANOVA (analysis of variance) analysis was used in this experiment to measure the significance level of difference of the post-test score and learning gain between the control group and KB group.We also measured the correlation between the map score and pre-test score, post-test score, and learning gain in the KB group to determine how close he concept mapping cognitive activity was with the learning achievement of the KB concept mapping activity.
International Journal of Information and Education Technology, Vol. 13, No. 9, September 2023

A. Experiment Results
The results for this experiment are shown in Fig. 5, which shows the test score visualization of the pre-and post-tests and the comparison of each group.Fig. 6 shows the learning gain value between the groups.A mean analysis was conducted for each group, and the result is shown in Table II.
To check the homogeneity of the participants, a Levene test on the pre-test results was performed, and the results are shown in Table II.In Table III, the Levene test for the homogeneity check on the pre-test resulted in a p-value of 0.8821 which was p > 0.05.This result concludes that the difference between the pre-test scores in the control group and pre-test scores in the KB group was not big enough to be statistically significant.In other words, both groups were homogeneous in terms of learning material knowledge.
In the mean analysis, which is shown in Table I, the post-test score of the control group was 62.3, whereas the KB group was 71.3.A Saphiro-Wilk normality test on the pre-test, post-test, and learning gain results was performed, and the result is shown in Table IV.All the p-values were <0.05 which shows a non-normal distribution.Therefore, a non-parametric Kruskal-Wallis ANOVA analysis was chosen for the next analysis.
To check the significance of the pre-and post-test scores from both groups, a non-parametric Kruskal-Wallis analysis was conducted, and the result is shown in Table IV.First, we analyzed the difference in the pre-test scores between the groups.We found that with a p-value of 0.5444, which is p > 0.05, the difference between the pre-test scores of both groups was not statistically significant.This means that participants in both groups have similar levels of knowledge.
After the activity, we analyzed the difference between the post-test scores in both groups.The p-value of 0.03461, which is p < 0.05, means that the post-test scores in the KB group were significantly higher than the control group.In other words, the KB concept map activity resulted in higher knowledge gain compared with the traditional summarizing method.
Learning gain was calculated using the normalized gain formula based on the pre-and post-test scores.The mean analysis showed that the KB group had a higher learning gain with a normalized gain value of 0.589 compared with the control group's normalized gain value of 0.425.The Kruskal-Wallis analysis was conducted on the learning gain value, and the results are shown in Table V.With a p-value of 0.009608 and p < 0.05, the difference in the learning gain between the control group and KB group is statistically significant.This means that learners could have a significantly higher learning gain with the KB concept map activity than with the traditional summarization method.The KB concept map tool provides an automatic assessment to check the learner map compared with the teacher goal map and calculates the map score.Using the Pearson R correlation, a correlation check between the map and pre-and post-test scores and learning gain was performed on the KB group.This check aims to get an insight into whether the KB concept mapping activity and the learner map score are correlated on the knowledge achievement and learning gain.The result of this correlation check is shown in Table VI.
The map score showed no correlation with the pre-test score with an r value of 0.215.The map and post-test scores had a high degree of correlation with an r value of 0.693.The map score and learning gain had a high degree of correlation with an r value of 0.669.This means that there is a relationship between the map and post-test scores and learning gain.Learners with a high map score tended to also have high post-test scores and learning gain.
The map and pre-test scores showed no correlation, but after KB usage, the post-test score was correlated with the map score.This is understandable because the map score was calculated after KB activity, and it was correlated with the level of learners' knowledge after the activity.This shows the usefulness of KB as an assessment tool.

B. Discussion
It can be concluded that with the addition of cognitive activity with KB as an interactive learning media, the learning outcomes of the essential facts and concepts of OOP could be improved in terms of the learning gain.In the traditional summarizing method, the learners were free to summarize any points from the learning materials, and they could do this any way they wanted.The only guidelines for the summary in this experiment were the provided keywords that was extracted from the learning materials.In the KB concept mapping technique, the learners needed to think more actively when creating propositions by connecting concepts and links to construct a complete concept map.The constraints were that they had to work on the concepts and links that were provided as -kit.‖Therefore, the contribution of this study is that providing learning media in the knowledge visualization for OOP when learning the outcomes of essential facts and concepts improves the learning gain compared with non-interactive media, such as the summarizing method.This study answered the RQ1.The correlation tests show that there is a strong correlation between the map and post-test scores and learning gain, which answered the RQ2.The visual learning strategy when implemented with KB concept mapping can be used as a better alternative when learning and visualizing the OOP concept compared with the traditional non-interactive presentation and summarizing learning method.

V. CONCLUSION AND FUTURE WORK
KB concept mapping can be a good way to provide knowledge visualization in the learning outcomes of OOP.In this study, the implementation of KB concept mapping on OOP learning outcomes placed in the category of essential facts and concepts of OOP.
This study found that using KB concept mapping can significantly increase the learning gain of OOP in students' learning compared with the summarization method.The KB concept map can also function as an assessment tool with its map score functionality.The map score was found to have a high correlation with the knowledge achievement level and learning gain.
The KB concept mapping system has been proven to improve the result of learning.It will be interesting to investigate the process of knowledge restructuring in terms of the concept mapping activity.The map score was found to have a high correlation with learning gain.It is possible to also measure the concept mapping process to achieve a better correlation measurement between concept mapping and learning gain.

Fig. 3 .
Fig. 3. Segmentation and structuring task in the KB framework.

Fig. 5 .
Fig. 5. Scores of the pre-test and post-test between groups.
describes the steps during each of the experiment sessions.

TABLE II :
MEANS OF THE PRE-TEST, POST-TEST, AND LEARNING GAIN

TABLE III :
LEVENE TEST FOR HOMOGENEITY CHECK ON PRE-TEST

TABLE V :
NON-PARAMETRIC KRUSKAL-WALLIS ANALYSIS ON POST-TEST AND LEARNING GAIN BETWEEN THE KB GROUP AND CONTROL GROUP

TABLE VI :
CORRELATION BETWEEN MAP SCORE AND PRE-TEST, POST-TEST, AND LEARNING GAIN SCORES