A Study of Phrase Fill-in-Blank Problem for Learning Basic C Programming

,


I. INTRODUCTION
Currently, in a lot of universities across the world, C programming is taught to novice students in the first programming course, before teaching more advanced and practical programming languages such as Java, Python, and JavaScript.Unfortunately, students are often suffering from studying it due to the formality nature of using symbols and logics programming.To assist their self-studies at homes without teachers, we have developed C programming learning assistant system (CPLAS) that offers several types of exercise problems with different leaning goals and levels.In any problem, an answer from a student is marked at the system automatically.
Previously, in CPLAS, we have defined and implemented the grammar-concept understanding problem (GUP) [1], the value trace problem (VTP) [2], the element fill-in-blank problem (EFP) [3], and the code completion problem (CCP) [4].In these problems, one problem instance consists of a source code, a set of questions, and their correct answers.The correctness of any answer is determined through string matching with the stored correct answer in the system.The common answer interface for them has been implemented on a web browser [5], where the marking function was implemented by JavaScript that runs on the web browser.
In a GUP instance, the source code contains basic keywords in C programming to be studied, such as reserved words and common library elements.Each question describes the grammar concept on a keyword and requests to answer the corresponding element appearing in the source code.It is noted that an element represents the least unit of the source code.GUP aims the grammar study.
In a VTP instance, the source code contains several standard output statements for important variables or output messages.Each question requests to answer the corresponding output value.VTP aims the code reading study.
In an EFP instance, the source code contains several blank elements with specifying the locations.An element can be a reserved word, an identifier, an operator, and a control symbol here.Each question requests to fill in the blank.EFP aims the code writing study.
In a CCP instance, the source code contains several blank elements like EFP, but does not specify the locations.Then, each question requests to find the location of a missing element in the code and fill in it with the proper one.CCP aims the code reading & writing study.
For any exercise problem type, every time a student submits the answer for an instance, it is stored in the web browser with the time stamp using Web Storage [6].When the student needs to submit the answers to a teacher, the answers by this student can be saved in a text file to be downloaded from the web browser.
After collecting the answers from the students, the teacher will analyze and tally them by running the answer analyzer.This Java program selects only the final answer of the student to each instance using the time stamp.Then, it calculates the average correct answer rate and the average number of submission times for each instance and each student from all the submitted answers by the students.After that, it will output them into the Excel file.
The existing problem types have different learning goals to assist step-by-step programming self-studies of students with CPLAS.Unfortunately, they are not sufficient for the code writing study.The question only asks to fill in one element in the source code.To complete a source code, it is necessary to make a combination of multiple elements properly.
Recently, as a new problem type for the code writing study of C programming by novice students, we have proposed and studied the phrase fill-in-blank problem (PFP) [7].In a PFP instance, the source code contains several blanks of phrases or sets of multiple elements.Each question requests to fill in the blank with the phrases that originally existed in the code.To generate a new PFP automatically as best as possible, we have implemented the PFP generator using Python.
In PFP, one blank can substitute any number of elements in the source code, unlike EFP or CCP where one blank can do only one element.This flexibility of PFP is able to increase the number of candidates or choices for blanks in the source code, which can enhance learning effects and difficulty levels at solving problems by novice students.
In this paper, we studied the phrase fill-in-blank problem (PFP) in CPLAS for learning basic C programming.Collecting proper source codes from [8][9][10][11][12] and running the PFP generator, we newly generated 13 PFP instances for learning basic grammar topics and nine instances for learning logic functions.Then, we assigned them with previously generated 15 instances for recursive functions, to 21 students in Okayama University.
For evaluations of the PFP instances, we collected the answer files from the students, and calculated the average correct rate and the number of submission times from them using the answer analyzer.Then, from them, we could confirm the correctness of the generated PFP instances and find the weakness of the students in C programming topics and the students to be cared.
The rest of this paper is organized as follows: Section II provides a review of related works in literature.Section III presents the PFP instances generated for this application.Section IV shows the example of PFP instance.Section V shows their application results to novice students and discussions.Finally, Section VI concludes this paper with future work.

II. RELATED WORKS
In this section, we discuss related works in literature.In [13], Freund et al. developed the Thetis programming environment that is designed specifically for student to use.This system consists of the C interpreter and the associated user interface to provide simple and easily-understood editing, debugging, and visualization capabilities.It is more suitable for students in introductory computer science courses, because some commercially available compilers, particularly those used for languages like ANSI C, assume the level of sophistication that novice students do not possess.
In [14], Boada et al. presented a web-based tool, named ACMEp, with the aim of reinforcing teaching and learning of introductory programming.From the teacher's perspective, this system can introduce important gains with respect to the classical teaching methodology where the teacher can perform continuous assessments of progresses of students.From the student's perspective, it provides a learning framework for programming and correction environments with helps, which facilitates their personal works.
In [15], Bravo et al. described several educational computer tools, named SICAS, PlanEdit, COLLEGE, and OOP-Anim, to support programming learning, and presented a global environment of integrating them, allowing a broader approach to programming teaching and learning.This environment offers animations and the computer-supported collaborative learning (CSCL) paradigm.In [16], Li et al. presented a game-based learning environment to support programming learning.It exploits game construction tasks to make the elementary programming more intuitive to learn, and comprises concept visualization techniques, to allow students to learn key concepts in programming through game object manipulation.

III. PHRASE FILL-IN-BLANK PROBLEM FOR BASIC C PROGRAMMING
In this section, we present the design goal and the generation of the phrase fill-in-blank problem (PFP) for learning basic C programming.

A. Design Goal of PFP
For the introductory coding study, EFP performs the limited programming activity of filling in only one element to each blank in the given source code.As the next step coding study, PFP is designed to work on the more realistic programming activity of filling in multiple elements together to one blank in the code.The number of elements in one blank has no limitation, as long as the blank elements belong to one statement in the code and can give the unique answer to the blank.These constraints come from string matching at the marking function.

B. PFP with Blank Phrases
For learning basic C programming efficiently, first, novice students should learn how to write standard input/output functions in the source code because they are necessary for visualizing the behaviors of the source code.Second, they should learn how to use the conditional statements using if for the control flow.Third, they should learn how to give the input and output of a function.By knowing the proper input/output of a function, it is expected that the whole source code can be understood.Fourth, they should learn the conditional statement for looping using while and for.Last, they should learn recursive functions and other important concepts in C programming.
In this paper, we generated 13 PFP instances for basic grammar concepts and nine instances for logic functions.
As summary, the following phrases or sets of elements are blanked from the given C source codes for basic grammar concepts: 1) the conditional statement of if and while 2) the statement of printf and scanf.
Then, the following phrases or sets of elements are additionally blanked from the source codes for logic functions, where 5) is added to allow a teacher to make blanks flexibly: 1) the conditional statement of for, 2) the argument of the function, 3) the returning phrase of return, 4) the header file, 5) the phrases selected by the teacher manually.

C. Alternative Answer
Unfortunately, the correct answer for a blank may not be unique.Besides the original phrase removed in the source code, other correct phrases may exist.They are called alternative answers in the paper, where the alternative answers are found and generated manually.
In a PFP instance, the blanked source code is shown to students who are requested to be filled in with the proper phases.The correctness of each student answer is marked through string matching with the original phase in the source code or the alternative answer if registered.

D. PFP Level
To make step-by-step studies through solving PFP instances at different levels, a teacher can select one of the following four levels at PFP instance generations: • Level 1: argument of printf, argument of scanf, • Level 2: (+) condition of if, • Level 3: (+) condition of while, argument of function, returning value of function, • Level 4: (+) condition of for, common library, other concept selected by teacher."(+)" indicates that the phrases in the lower levels are included at that level.Level 1 considers only the standard input and output.Level 2 considers the if condition in addition to Level 1. Level 3 considers the while condition, and the argument and the returning value of the function, in addition to Level 2. Level 4 considers the for loop condition and the common library, in addition to Level 3. Here, a teacher can choose any phrase in the source code if necessary.

E. Code Specification
In a PFP instance, the code specification is given together when it is necessary to reach the unique correct answer.For example, to answer the argument of printf, the actual standard output message is necessary.Also, to answer the conditional statement of while, the corresponding behavior of the code should be described there.

F. Generation of PFP Instance
A new PFP instance can be generated by the following procedure: 1) To select a proper source code that includes the programming concepts to be studied by this instance, 2) To select one PFP level, 3) To run the PFP generator which is implemented by python, to automatically find and remove the corresponding phrases in the code to this level for the blanked source code, keep the removed phrases for the correct answers, add the alternative answer phrases for the correct answers if necessary, and combine them into one text file, 4) To run the answer interface generator in [5] with this text file, to generate the CSS/HTML/JavaScript files for the answer interface on the web browser, 5) To add the code specifications into the HTML file, if necessary to reach the correct answers, 6) To register the generated PFP instance as an assignment to the students.

IV. EXAMPLE PFP INSTANCE
In this section, we discuss the example PFP instance where Level 3 is selected.

A. Example Source Code
BOX1 shows the example source code that accepts four positive integers from the standard input and outputs the maximum value among them.This code is used for the first PFP instance (ID=1) for learning basic grammar concepts.

B. Text File to Answer Interface Generator
BOX2 shows the corresponding input text file to the answer interface generator for this PFP instance.The conditional statements of if and while, and the statements of printf and scanf are blanked.The correct answers are stored in the system when they are blanked from the source code automatically.The correct answer for each blank is separated by ",,".Since the system can register up to two correct answers for each blank to consider alternative answers, the two answers are separated by "---" if exist.In this instance, value>max and max<value are registered as the correct answers for ( _7_

D. Answer Interface
Fig. 1 shows the answer interface to this example PFP instance.The left side shows the source code with the input forms to be filled in.When a student clicks the "Answer" button, the correctness of the input answers is checked through string matching, and the background of the incorrect answer becomes red while the correct one keeps white.In this figure, the incorrect answer should be corrected to value<=0.The right side shows the code specification, which is given to solve this PFP instances.

E. Data Analysis Program
After a student completes answering the given PFP instances, he/she will submit the answer text file.BOX4 shows the example.The first column in the file represents the student ID number.The second column does the PFP instance ID.The third and fourth columns do the submission date and time.The last column does the submitted answers and their making results, where those for each question are separated by ",".[o] indicates the correct answer and [x] does the wrong one respectively.Since any submission record is stored in the text file, it becomes difficult for a student to copy the file of other student and submit it to the teacher.
After collecting the answers from the students by emails or a file server, the teacher will calculate the average correct answer rate and the average number of submission times by running the answer analyzer and analyze them using the output Excel file.In this section, we discuss the evaluation of the phrase fillin-blank problem (PFP) for learning basic C programming.We generated PFP instances at Level 3 or Level 4 for learning basic grammar topics, logic functions and recursive functions, and assigned them to undergraduate students in Okayama University, Japan.

A. Evaluation for Basic Grammar Topics
First, we discuss the evaluation of PFP instances for basic grammar topics.

1) Generated PFP instances
Table I shows the instance ID, the program topic, the number of lines in the source code (LOC), and the number of answer forms in each of the 13 PFP instances.2) Solution results of individual instances Fig. 2 shows the average number of answer submission times and the average correct answer rate (%) by all the 21 students of each of the 13 PFP instances.The instance at ID=10 gave the lowest correct rate and highest submission time, which suggests that array was the hardest basic grammar topic for beginners.The instance at ID=13 showed the highest correct answer rate, because this instance asked only the printf statement.3) Solution results of individual students Fig. 3 shows the average number of answer submission times and the average correct answer rate among the 13 instances of each student.The 11 students at ID=2, 3,4,6,8,10,11,17,18,19 and 21 achieved the 100% correct rate for any instance.The student at ID=14 took the highest submission times, which means this student solved them seriously but needs improvements in C programming.The two students at ID=12 and 15 showed the low correct rates 43.94% and 18.18%.They need more efforts in studying basic grammar topics of C programming, which should be watched and cared by the teacher.

B. Evaluation for Logic Functions
Next, we discuss the evaluation of PFP instances for logic functions.

1) Generated PFP instances
Table II shows the instance ID, the logic function topic, the number of lines in the source code, and the number of answer forms for each of the nine PFP instances.2) Solution results of individual instances Fig. 4 shows the average number of answer submission times and the average correct answer rate (%) by all the students for each instance.The instance at ID=22 results in the lowest correct rate.Memory allocation will be the difficult function for them, where it needs to understand the malloc and free functions.The instance at ID=20 took the highest submission times, where it has several blanks for if and while statements in the recursive function.
3) Solution results of individual students Fig. 5 shows the average number of answer submission times and the average correct answer rate among the nine instances for each student.The five students at ID=1, 3, 4, 18, and 19 achieved 100% correct rate.Again, the same students at ID=12 and 15 showed the low correct rates and the low submission times.

C. Evaluation for Recursive Function
Finally, we review the evaluation of the PFP instances for recursive functions in [7].

1) Generated PFP instances
Table III shows the instance ID, the recursive function topic, the number of lines in the source code, and the number of answer forms for each instance.2) Solution results of individual instances Fig. 6 shows the average number of answer submission times and the average correct answer rate (%) by all the students for each of the 15 instances.The instance at ID=35 exhibited the lowest correct rate.Tower of Hanoi can be difficult for the students because it recursively calls the same function twice with the different arguments.The instance at ID=37 showed the highest submission times, where they may be not familiar to the source code of the Ackermann Function using the recursive function.3) Solution results of individual instances Fig. 7 shows the average number of answer submission times and the average correct answer rate among the 15 instances by each student.The four students at ID=3, 4, 18, and 19 achieved the 100% correct rate, and the seven students at ID=5, 8, 12, 14, 15, and 17 did under 80%.programming course?For the first question, the answer is yes, because some students solved all the questions correctly.For the second question, the answer can be yes or no, because the correct rates in many instances exceed 80% whereas in some PFP instances, they were lower than 70%.It will be necessary to give hints for hard questions that cannot be solved even after several submissions, which will be in future works.For the last question, the answer is yes, because the two students at ID=12 and 15 showed the very low correct rates and submission times.

VI. CONCLUSION
This paper studied the phrase fill-in-blank problem (PFP) for learning basic C programming.13 instances for basic grammar topics, nine instances for logic functions, and 15 instances for recursive functions were generated using C source codes containing the corresponding concepts.The application results of the 37 PFP instances to 21 undergraduate students confirmed the correctness of the generated instances and found the weakness of students in C programming study.In future works, we will implement hint functions for hard PFP instances, generate new PFP instances for other topics in C programming, and evaluate them through applications to novice students.

Fig. 2 .
Fig. 2. Solution results of individual instances for basic grammar topics.

Fig. 3 .
Fig. 3. Solution results of individual students for basic grammar topics.

Fig. 4 .
Fig. 4. Solution results of individual instances for logic functions.

Fig. 5 .
Fig. 5. Solution results of individual students for logic functions.

Fig. 6 .
Fig. 6.Solution results of individual instances for recursive functions.

Fig. 7 .
Fig. 7. Solution results of individual students for recursive functions.
BOX3 shows the code specification for this PFP instance.To describe the output messages of the code, it includes the International Journal of Information and Education Technology, Vol. 13, No. 9, September 2023 standard input/output example when the code is executed.BOX3: Code specification Create a program that inputs four positive integer values and outputs the largest value.However, if a negative value is entered, re-enter it.An example of output:

TABLE I :
PFP INSTANCES FOR BASIC GRAMMAR TOPICS

TABLE II :
PFP INSTANCES FOR LOGIC FUNCTIONS

TABLE III :
PFP INSTANCES FOR RECURSIVE FUNCTIONS