Utilizing Graphical Elements for Concept Map Analysis to Design Teaching and Learning Assessment

The graphical elements as parts of concept map construction are employed to assess both learning and teaching. Augmenting the use of concept maps, this study examines the graphical elements, such as, nodes, edges, cliques, diameters, travelling paths and structures of the graphs to relate to ones’ understanding to a topic, in this case, polynomials for middle school. In the aspect of teaching assessment, the teacher’ concept map drawn according to the lesson plan is served as the master map, which echoes the teacher’s expectation of students’ learning. On the other hand, students’ maps also reveal their understanding through the nodal relationship, which can be the definitions of terms, related examples, graph representation and algebraic manipulation. Data collection includes a focus group of 10 students and 1 teacher undergoing the concept map assessment task with restricted node terms. Graphically analyzed, students’ concept maps reveal some common elements as in the teacher’s map. In addition, the interview with the teacher also suggests that concept map as the assessment tool is an effective teaching reflection for which the teacher can see what to fulfill for future classes.


I. INTRODUCTION
Concept maps have received a great deal of attention in science education as a learning strategy since 1990 [1]. The continuing research about this further suggests that the concept map construction is based on the epistemological assumption that the concept-concept relationship is the building block of knowledge [2]. However, structuring knowledge can be diverse, yet hard to visualize, particularly for abstract mathematical concepts. Therefore, use of concept mapping becomes handy to extract learners" knowledge construction and to reflect their understanding. In accordance with this argument, McGowen and Tall claims that a concept map is a diagram representing the conceptual structure of a subject discipline as a graph in which nodes represent concepts and lines connecting them represent cognitive links [3]. With the beneficial structures, some research extends the application of concept maps as an assessment tool, for which the comparison between a teacher and students" concept maps can be considered as a form of a lesson evaluation to assess the lesson"s objectives [4], [5]. According to multiple reviews, the previous work about concept mapping assessment has involved around concept recognition, organization of concept in branching structure, graph alignment, similarity, and scoring scheme for learning Manuscript received March 17, 2020; revised June 10, 2020. The authors are with Mahidol University, Thailand (e-mail: chuechote@gmail.com). assessment [6]- [10]. Yin et al."s research offers the comparison between two models of a concept map construction; one model is a concept map construction assignment with only node terms restricted-students self-create linking phrases-and another is just a map assembling task where both node terms and linking phrases are provided [10]. From this work, it suggests that the first model is better for knowledge capturing, whereas the latter fits better for large scale scoring. Clearly seen in the map assembly task, the scoring could be bipolar; matching is either right or wrong. On the other hand, in the open-ended approach, the constructed concept maps can diversify and generate more complication in scoring. Since this study emphasizes on learning assessment, we therefore have embraced the first model of concept map construction with node terms about polynomials provided. To tackle with the complication of scoring, we have explored the graphical elements and features, such as, nodes, edges, diameters, cliques, travelling paths and structures, for potential use in map scoring scheme. With awareness of reasoning behind the map construction process, the collection of concept maps from students will be compared with the teacher"s map to make a better vision of how the teacher has expected and what students have achieved.
Analyzing graphical data, we have considered how these graphs are formed and how the elements are linked. The graphical result and its interpretation provide interesting angle in learning and teaching assessment. Remarkably, concept map similarity among students engaging in the same lesson can mirror prior knowledge of students and the effectiveness of the teaching approach. This research gives another purposeful use of the concept maps and suggests diagnostic scheme that could be beneficial to both learning and teaching assessment.

II. CONCEPT MAP CONSTRUCTION
Since knowledge construction process is the integral part for this study, we designed the experiment to include 3-hour training of concept map construction following the adapted scheme from Malone & Dekkers [11]. The protocol is described as follows.
-First task: List the key terms and find all possible terms that can be associated with the key terms.
-Second task: Rank the strength in association of the key terms and other node terms. Arrange the key terms on top and the closest or strongly associated node terms with the key terms are one level below. If node terms have the same relationship with the key terms, put them on the same level.
-Third task: Add edges or linking phrases according to the relationship.
-Fourth task: Look for relationship between key terms or between node terms and add edges. In this study, we have explored concept mapping on polynomials topic in middle schools. This topic has a potential for capturing students" geometrical and algebraic reasoning as well as reflecting teaching approach and media used in the lesson. Therefore, to teach this topic, the teacher should prepare the plan that give students enough experience for relevant knowledge.
To design the set of node terms, we considered possible objectives that the teacher aimed to achieve for her teaching. The node terms were deduced from the teacher"s lesson plans, learning objectives and pre-lesson interview. The declared objectives were to understand the meaning of terms, monomial, polynomial, similar monomial; to be able to manipulate, add, subtract, multiply and divide polynomials; to be able to apply it with algebraic reasoning, and to understand graphical interpretation. Fig. 1 shows the node terms used in the concept mapping assessment. The 7 categories of node terms are denoted as; "A" for definite terminologies, "B" for algebraic expressions, "C" for short description, "L" for graphs, "H" for area and volume, "K" for algebra tiles representation, and "Q" for polynomial factorization. To avoid bias in concept mapping caused by the teacher execution, these terms were not shown to the participants until the day of assessment. Clearly illustrated this protocol, Fig. 2 shows how one works on concept map construction following the protocol. The components on cognitive development, such as concepts recognition, grouping concepts and organizing concepts in branching are visible via the process of concept maps construction. After being familiar with this, the participants attended their regular mathematics class which covered a polynomials topic and were back to do concept mapping assessment as a major part of data collection. By the end of this process, we obtained 11 concept maps from both teacher and student participants and the teacher interview on her teaching approach, teaching media, styles and difficulties she experienced. The concept map built by the teacher served as the master map. It implied the expected leaning outcomes. Hence, it was used for reference in the graph alignment, reflecting how well the teacher delivered the domain content and concept.

III. CHARACTERISTICS OF A POLYNOMIAL CONCEPT MAP
A concept map is a diagram that represents conceptual structure of a subject discipline as a graph G = (V, E) as a pair of a set of vertices (or nodes) V and a set of edges E. The edges represent relationships between the nodes. They can be oriented or not, depending on the nature of relations represented. Every graph is described by connecting nodes, which can be written in the form of an adjacency matrix A, a binary n × n matrix with entry a ij = 1 if node v i is adjacent to node v j , and a ij = 0 otherwise. The number of nodes in a graph is usually denoted by n while the number of edges is usually denoted m. The diameter of a graph is the longest shortest path between any pairs of nodes. In other words, a diameter is the largest number of edges which must be traversed in order to achieve shortest distance travel from one node to another node. According to the graph as in Fig. 2, we have the adjacency matrix A as 8 × 8 matrix, shown in Fig. 3 for the undirected graph. With diameter of the graph equal to 5, the longest shortest distance of the graph is the path travelling from "8x 2 +5x" to "-x 3 ". In this matrix form, we can easily manage graphical comparison. When each matrix entry represents node terms" relation, the frequency of edge links, cliques, common subgraphs or travelling paths can be computed with International Journal of Information and Education Technology, Vol. 10, No. 8, August 2020 the aid of R programing and its functions for matrices.
A subgraph of a graph G is a graph whose nodes and edges are contained in G. A graph in which all nodes are adjacent to all others is said to be complete. A clique is a maximal complete subgraph of G. In other words, a clique of an undirected graph is a subgraph whose every two distinct nodes in the clique are adjacent.
These graph elements and features play the important role in concept map analysis. With the assumption that students undergo the same process of the lesson, they likely generate common meaningful subgraphs. The common subgraphs can be in several forms, a net, a tree, a string or a clique. With the unique property that any nodes are adjacent, a clique then has potential to present a knowledge preposition. The investigation of knowledge formation representing through the graph will be delineated in the next section. However, with the master map from the teacher, a student"s map will then be compared with map alignment for a similarity score, the comparison of adjacency matrices between teacher"s and a student"s map. Map similarity has been widely researched in computer network communities as they have similar structural features, for example, web and internet uses, semantic network, and accessibility. Likewise, the collection of concept maps as teaching and learning assessment can also be organized in the ontology framework using graphical features and hence become useful for the analysis of the domain of interest in the similar fashion [12], [13].

IV. METHODOLOGY
The aim of the research was to explore concept maps drawn by a teacher and students with the assumption that the maps" elements and features would show relatedness to students and teacher"s understanding and hence navigate to graphical interpretation that reflects teaching and learning. The research involved the focus group containing 11 members, 1 teacher and 10 middle-school students, from Phitsanulok province, Thailand. There was a training phase to make sure that all participants were able to draw concept maps when node terms were provided. Note that the training phase we worked on the primary geometry topic to avoid pre-test manipulation bias.
In addition to students" concept maps drawn after the lesson finished, the data collection included pre-and post-lesson interview with the teacher, a teacher"s lesson plan for polynomials topic, a teacher"s concept map drawn after lesson implementation. The interview questions were designed to go into details of the teacher"s lesson plan, teaching approach, expectation, reflection and difficulties after lesson implementation.
The analysis principally involved the graphical data, pattern extraction and graph alignment for class comparison and teacher-student comparison. The teacher"s lesson plan and interview were to support the graphical interpretation and self-reflection on the lesson execution.
V. GRAPHICAL ANALYSIS When students are asked to draw concept maps regardless of any intervention, it is likely to see different maps explaining the same concept but with different structure representation. A teacher could find it difficult to check whether the students have understood and have met the objectives of the lesson. Therefore, we consider the graphical elements and features in association with the learning domain. In this case, it is polynomials topic for middle school mathematics.

A. Key Terms and Their Association
Concept mapping as a learning assessment has a key term where other terms are built around. At this section, we capture the core understanding of the topic indicated by the key terms and their neighbors. According to section III, the concept map construction protocol, a key term is supposed to get listed first with potential of being the hub. A sketch of knowledge prepositions can be deduced from the neighbors of a key term. To capture this individual latent understanding, we therefore associate the linking edges according to the cognitive links into 5 types; (1) defining, (2) containing, (3) example, (4) graphing, and (5) algebraic manipulating, as shown in Table I. Fig. 4 shows the possible concept map whose edges between nodes are categorized based on the types of nodal association that could infer the comprehension and ability to think around the key terms of the topic.

B. Path Depth with Cognitive Reasoning
Investigating semantic graph in the aspect of knowledge formation, researchers in graph mining community extend a graph depth of each entity to be a means of similarity measurement but with corresponding weight according to their importance in the domain [11]. According to the learning objectives of the polynomials lesson, we have investigated the cognitive reasoning through the paths that pass the key terms. Therefore, meaningfulness of a concept map in learning must consider the graphical path depth and the combination of linking edge types. In this graphical dataset, the diameter of a student"s concept map contains the longest path of linking node terms. This path in Fig. 5 shows how one student represented his reasons through the nodal relationship. Constructing a concept-centered map with a key term put at the top level, the participants associated neighbor terms ranked by closeness according to their understanding. The diameter, which is defined as the longest path in each graph, may have diverse linking phrases and edge types. This echoes how ones can understand a concept, can define, can International Journal of Information and Education Technology, Vol. 10, No. 8, August 2020 give examples, can apply, and/or can communicate via graphical representation. Fig. 5 illustrates an example of student"s concept map. By reading the map, we could see that this student understands that a polynomial (A4) can contain variables (A1) and can be defined as addition of monomials (C4), which can be exemplified as in (B18), algebraically manipulated to be equal to (B7). The diameter in Fig. 4 passing through the key term, "polynomial" contain 4 out of 5 edge types. This information signifies the cognitive reasoning revolving around the key conceptual term. The path depth with various edge types about the key terms identifies the understanding in various dimension that could eventually satisfy the learning objectives.

C. Cliques and Concept Formation
Definition of a clique already suggests that any two node terms are related. For a polynomials topic, the node terms can be linked to form a clique and that clique could represent a knowledge preposition. Fig. 5 shows that magnified clique of the graph contains 4 nodes, "polynomial (A4)", "addition of monomials (C4)", "addition of similar monomials (C5)", and "5a 2 b 3 +b 3 a 2 (B18)". Considering this 4-clique, we see that the formation that links the definition of the term "polynomial", its characteristics, and its example, which is extended to reveal the ability of algebraic manipulation and understanding of "similar monomial". This graph also shows interesting 3-clique, containing 3 nodes "45 (B4)", "Coefficient (A2)", the graph of constant in the xy-plane (L3). The interesting point here is that this student shows his knowledge indicating the misconception, and possibly the area that the teacher has neglected while teaching this topic. He shows that he understands graphical interpretation of a constant but has a misconception about coefficient. The coefficient multiplying a variable can produce various graphs, not just a horizontal line. From this evidence, the teacher can see the misconception and can correct it in the future.

D. Graph Alignment
Graph matching takes all the attributes of nodes and edges arranged in the form of an adjacency matrix. We first used matrix comparison to see the match of each entry of the compared matrices. Fig. 6 shows the union of 10 students" concept maps with yellow-highlighted nodes, in comparison to the master map constructed by the teacher. Therefore, the non-highlighted node terms represent the areas that teacher over-expected; no students shows the relation of terms such as "area (H1)", "volume (H2)", "a 2 +b 2 +2ab (Q2)". On the other hand, Fig. 7 reveals the most common students" concept maps, i.e. 5 out of 10 students has got this subgraph. Also compared with Fig. 6, there is some room of improvement where students need to practice and learn more to achieve the understanding of the white terms" concepts. In this case, the node terms of categories "H" for area and volume, "K" for algebra tiles representation, and "Q" for polynomial factorization were left undone by the students. The teacher reflected herself that she should have used algebra tiles to be the manipulative for this lesson so that students could visualize the factorization of polynomials and better understand how polynomials related to areas and volumes and other applications.  However, with the maps reflecting conceptual understanding, graphing alignment alone is insufficient to International Journal of Information and Education Technology, Vol. 10, No. 8, August 2020 assess conceptualization, particularly, thinking about polynomial and factorization concepts. The alignment might give a high score if a student selects a key term as a nodal hub with its neighbor nodes as exemplifying terms, for example, "monomial (A3)" acting as a hub with 9 neighbor terms of its examples, such as, "9x 2 ", "-16x", etc. This matching subgraph does not show other cognitive reasonings, such as, algebraic manipulation, concept connection or graph interpretation. Therefore, graph alignment can only measure similarity to the teacher"s master map but not the growth of learning as an individual reveals the connection formed with various reasonings to show how he or she conceptualizes.

VI. RESULTS AND DISCUSSION
We started the work with a small focus group to make sure that all participants could draw the map. Hence, it eliminated the interpretation of not drawing because of how-to issue. With the in-depth interview with the teacher prior to the polynomial lesson, we opened her to implement the lesson in the way she was confident. She rated herself as an active questioning teacher, who rarely used thinking manipulatives but the effective questions to motivate students" thinking. She believed that math exercise would help students understand. Therefore, in her lesson plans, she put the assignments corresponding to her lesson objectives. Considering concept maps of students, she then reflected herself that it would be better to blend in different approaches for her class. Fig. 8 shows the edge types found in the teacher"s map and average of students" maps. From this evidence, it shows that most students can only obtain 41.67% of the master map. In other words, the average map from the students aligned 41.67% of all components of the master map, so called it similarity score. The lowest similarity score is about 20% and the highest similarity score is 85.19%, shown in Fig. 9. Interestingly, Fig. 8 also tells that the edge types that students correctly linked were defining and containing. Agreeing with the graphical analysis, the teacher recorded in her lesson plan that she assigned exercise for the students to describe the definition of the polynomial and monomial terms. Not being in the major part of the plan, the graphing and algebraic manipulating were the areas that students failed to achieve. The teacher mentioned that she did not emphasize on graph representation because it was not quite associated with the topic. However, relating polynomials with graphs, starting from polynomials of degree 1 or linear relation will benefit the students in graphical interpretation and hence lead to spatial skills for higher-order problems. Nonetheless, the teacher agreed that having the algebra tiles for the manipulative of this topic would be appropriate. Some students would learn better with visualization and hands-on. If she had done this, the edge types of graph and algebraic manipulation found would become close to the master map. For the advantage of students, hands-on activity, specially the algebra tiles as for algebraic manipulation, would cultivate the sense of algebra and geometry linked and future concepts for factorization Fig. 9. The similarity score and percentage of cliques found in concept maps of 10 students.
According to the master map constructed by the teacher, a similarity score is the percentage of nodes and edges from a student"s map aligned with the master map. As mentioned earlier, this score is not the best representation of conceptual understanding. Some students may have high similarity score but fail to capture the core concept. As Fig. 9 shows the student coded FG1-03 has 40.74% as his similarity score but only discovered 2 cliques. On the other hand, the student coded FG1-04 has lower similarity score 37.04% but dis covered 3 cliques, counted as 25% of the master map"s cliques. Having cliques in the concept maps suggests the understanding in more complex relationship than the linear links or maps. As for example, a clique of nodes "polynomial (A4)", "addition of monomials (C4)", "addition of similar monomials (C5)", and "5a 2 b 3 +b 3 a 2 (B18)" shows the links between any pairs with different justification levels. This net-like structure of a clique incorporates a conceptual meaning with the map integrity. It infers the builder"s understanding around the domain concept with his/her ability to employ technical terminology [9], [14].

VII. CONCLUSION
The comments of the teacher after seeing the result of concept map analysis marked the significant self-reflection that led to the promising action plan. Therefore, we have a good start and would like to move this study forward for concept maps collection and graph mining. In addition, developing the electronic version for concept map construction will be useful. It will aid students to draw concept maps; pasting nodes and making links can be done more easily. Similarly, teachers can choose the significant International Journal of Information and Education Technology, Vol. 10, No. 8, August 2020 node terms corresponding to their lesson objectives to assess students" learning.
Considering the graphical elements and feature, we have seen the promising relation of a map"s structure and knowledge construction. Therefore, the scoring scheme of a concept map should incorporate the interpretation of the elements and features like diameters, cliques, and graph structures to the similarity score. As Kinchin and Hay suggested, concept mapping can help us see student understanding more explicitly. A student with a net-like map structure or with many cliques tends to show more various levels of thinking than the one with a linear map [9], [14].
This study serves as the initial phase to understand graphical data as for psychological and cognitive measurement. The research team will explore further for the best fit scoring scheme and graphical database for concept mapping. The reflection from the teacher encourages us to investigate more with various learning topics, various students" backgrounds and various learning objectives as we have been aware that teaching approach plays role in students" concept map construction. When it comes to learning and teaching, it needs to be locally customized. The ideal assessment for learning should be a means that can extract students" knowledge construction and reasoning trajectories. Whereas, the assessment for teaching should feedback teachers for their past actions and should shape or navigate to the teaching improvement to achieve students" learning. This research has achieved making the participating teacher realized that.