A Proposal of Hint Function for Java Programming Learning Assistant System

 Abstract —Nowadays, Java is one of the world’s most used object-oriented programming languages for its dependability and portability. To assist the self-studies of novice students, we have developed the Java programming learning assistance system (JPLAS). JPLAS offers several types of exercise problems for different difficulties, including the grammar-concept understanding problem (GUP) , the mistake correction problem (MCP) , the element fill-in-blank problem (EFP) , and the phrase fill-in-blank problem (PFP) , where a question asks to answer the corresponding keyword or phrase in the given source code. Unfortunately, some students cannot solve them well as the difficulty level is ascending. In this paper, to help such students, we propose a hint function that will show the first or more characters of each correct answer word in the answer interface on a web browser for JPLAS, when requested. The use of this function by a student can be traced by a teacher to analyze the performance and problem difficulty. For evaluations, we generated three instances for GUP, MCP, EFP, and PFP respectively, and assigned them to junior students taking the Java programming course in Okayama University, Japan. Their answer results found that the proposal is effective in helping students solve exercise problems in JPLAS, but the thinking time needs to be adjusted for PFP.


I. INTRODUCTION
For decades, Java has been widely used in industries as the dependable and portable object-oriented programming language.Java has also been implemented in mission-critical systems for major organizations as well as in small-sized embedded systems.Thus, there has been a high demand among IT firms on educations of Java programming designers.Actually, a great number of universities and professional schools have offered Java programming courses.
To enhance Java programming educations, we have developed a web-based Java programming learning assistant system (JPLAS) [1,2], JPLAS provides a variety of exercise problems that may be used to gradually progress the learning stages and cover self-studies of Java programming at various levels by novice students.In any instance of these problems, an answer from a student is automatically marked in the system.Therefore, a student can continue solving the given problems until all the answers become correct.
Actually, JPLAS offers the grammar-concept understanding problem (GUP) [3], the mistake correction problem (MCP) [4], the value trace problem (VTP) [5], the element fill-in blank problem (EFP) [6], the code completion problem (CCP) [7], the phrase fill-in-blank problem (PFP) [8], and the code writing problem (CWP) [9].In any instance of these problems, the correctness of an answer from a student is automatically checked in JPLAS.For GUP, MCP, VTP, EFP, and CCP, it is checked through string matching with the correct one stored in the system.For CWP, it is verified through unit testing by running the test code on JUnit.Among these problems, this paper is targeting GUP, MCP, EFP, and PFP in JPLAS, since a new instance can be generated automatically by running our implemented instance generator, once the proper source code is selected.
The difficulty level will be increased as this order.Therefore, we have found that more students cannot solve PFP in previous implementations to them.To make them continue solving the questions, some assistance functions will be necessary when they meet difficulties.
In this paper, we propose and implement a hint function in the answer interface on a web browser for the four problems of GUP, MCP, EFP, and PFP in JPLAS.This function will show the first character of the correct answer word for the question where the answer of the student is not correct, when he/she requests it by clicking the corresponding button.Besides, it will show the correct answers to the questions in the instance, if the student cannot reach them after a lot of trials.
For PFP, our previous applications to students found that the first character of the answer word is not a sufficient hint for a student to reach the correct answer.A question in PFP asks multiple words called the phrase at once for each question.Therefore, we modify the hint function to make two steps.In the first step, the first character of the correct answer is shown.Besides, the other characters are also shown after being replaced by * to hide them, so that a student can know the number of characters in the answer.In the second step, * is replaced by the corresponding character in the correct answer randomly with 30%.Therefore, approximately, one-third of the answer word can be noticed by the student.
To avoid a student relying on the hint function from the beginning, the hint button can appear only after he/she clicks the answer button five times and five minutes have passed since the page is accessed.
To allow a teacher to know whether each student used the hint function at each instance, its use will be recorded and be written as the hint function flag in the answer text file that will be submitted to a teacher for grading.By checking the flag, the teacher can evaluate the performance of each student by considering the use of the hint function.The teacher can also estimate the difficulty level of each instance by counting the number of students who used the hint function.
To implement the proposed hint function in the answer interface, we extended the answer interface generator so that the generated HTML/CSS/JavaScript files include the necessary button, the output field, and the hint display function.Besides, we extended the student answer analyzer, which will make the summary of solving results from the answer text files submitted by students, considering the use of the hint function by the individual students.
For evaluations, we generated three instances for GUP, MCP, EFP, and PFP individually, and assigned them to junior students taking the Java programming course in Okayama University, Japan.Their answer results found that the proposal is effective in helping students solving exercise problems in JPLAS, but the solving time needs to be adjusted for PFP.
In addition, we present a hint function for code completion problem (CCP) by extending the one for PFP.A question in CCP asks to complete the given source code that has several blank elements and incorrect ones.At the answer marking, each whole statement is compared with the corresponding correct one through string matching.The evaluation of the hint function for CCP will be in future studies.
The rest of this paper is organized as follows: Section II discusses related works in literature.Section III reviews our preliminary works on PLAS.Section IV presents the hint function for four problem types.Section V evaluates the hint function.Section VI presents the hint function for CCP.Finally, Section VII concludes this paper with future works.

II. RELATED WORKS IN LITERATURE
In this section, we discuss related works in literature on the hint function in programming by novice students.
Yi et al. [10] implemented a new repair strategy in automated program repair (APR), which is similar to the hint generation policy used in the previous intelligent tutoring system for programming (ITSP).This new policy allows partial corrections that address a portion of failed tests.However, novice students do not know how to effectively use the generated repairs as hints.
Marwan et al. [11] suggested that a growing body of works has explored how to automatically generate hints for novice programmers.They explored the efficacy of next-step code hints that have two complementary features: textual explanations and self-explanation prompts.They discovered that code hints with textual explanations dramatically increased their immediate programming performances.
Price et al. [12] presented the QualityScore program as a new method for automatically assessing and comparing the quality of the next programming hints using expert ratings, whereas few assessments directly evaluate or compare quality of different hint generation methods.The QualityScore program was used to compare the quality of the six data-driven next hint generation algorithms.The results show that although there are significant differences in quality between the six algorithms, they are relatively consistent across the different data sets and problems.
Rubinstein et al. [13] employed a machine learning platform called Sense Education in the introduction CS course.It delivers hints in real time while students work on solutions, as well as detailed feedback on submissions.It also gives a "bird's eye" picture of current capabilities and misconceptions of the class, and is biased in problem-solving approaches to train teachers how to use these technologies successfully.
Serth et al. [14] discussed the findings from providing contextual recommendations in a web-based development environment used for practical programming activities.They discovered that individuals requesting tips took substantially longer time and used the platform's assistance features more than the users in the control group.The findings can be used to make more particular hints to beginners, and provide additional and context-specific clues as part of the learning materials.
Fein et al. [15] suggested that when learning programming, to take the first step can be difficult.It is desirable for a teacher to use a system that automatically generates prompts to tell them what steps to take in the next programming task.Catnip is the tool that generates next step prompts for Scratch programming.Catnip uses extensive post-processing to improve the generated prompts, and displays them directly within the Scratch framework.

III. REVIEW OF PLAS
In this section, we review the answer platform, and the four exercise problems of GUP, MCP, EFP, and PFP as our preliminary works on JPLAS.

A. Overview of JPLAS Platform
The answer interface platform for JPLAS was newly implemented using Node.js[16] as the web application server the under the uniform design [2].Node.js allows implementations of the application programs for both the server and client sides using JavaScript.Express.js[17] and EJS [18] are adopted together.Besides, Docker [19] is used for easy and correct distributions of the platform to students.
As programming languages for the implementation of the MVC model-based web application system, Java is used for the model (M) to run JUnit for testing the answer source codes from students, EJS/CSS/JavaScript are for view (V), and JavaScript is for controller (C), as illustrated in Fig. 1.The file system is directly used for managing data, where no database system is included.

B. Grammar-Concept Understanding Problem (GUP)
A question in a GUP instance requests to answer an important word and a common library class/function in the programming language as the first-step study of basic grammar concepts.Fig. 2 illustrates the answer interface for an example GUP instance.The source code checks if the given year is a leap year or not by using a if else statement.A student is requested to answer the element in the source code whose definition is described in the question.

C. Mistake Correction Problem (MCP)
A question in a MCP instance requests to answer each mistaken element and its correction in the given corrupt source code for code debugging study.Fig. 3 shows the answer interface for an example MCP instance.The source code in the left column has four mistakes: byte at line 3 should be int, / at line 5 be %, == at line 9 be !=, and retrun at line 10 should be return.In the right column, the corresponding line number is shown with each pair of input forms to answer the mistaken element and the correct element respectively.

D. Element Fill-in Blank Problem (EFP)
A question in a EFP instance requests to fill in the blank elements in the following source code with their originals by understanding the syntax and semantics.Fig. 4 shows the answer interface for an example EFP instance.In this source code, the while loop is iterated until the test expression num != 0 is evaluated to be false.Following the initial iteration, the value of num will be 345 after being divided by 10.The count is then increased by 1.After the second iteration, the value of num will be 34 and the count is incremented to 2. After the third iteration, the value of num will be 3 and the count is incremented to 3.After the fourth iteration, the value of num will be 0 and the count is incremented to 4. The loop ends here because the test expression is determined to be false.A student is requested to fill in each blank by carefully reading the source code.

E. Phrase Fill-in-Blank Problem (PFP)
A question in a PFP instance requests to answer the corresponding phrases in the vacant part of the source code to make it complete.Fig. 5 illustrates the answer interface for an example PFP instance.The source code in the left column checks if a given string or number is palindrome or not.A string is considered to be a palindrome if both are the same when it is read from left to right or read from right to left.The source code first reverses the string or number.Then, it compares the reversed ones with the original value.The string Radar is stored in str at last.A student is requested to fill in each blank by carefully reading the source code and the statements in the right column.

F. Instance Generation and Solution Steps
A new instance is generated and is solved through the following steps: 1) Collect the proper source codes of Java programming from textbooks.2) Read a Java programming source code file.
3) Run the generation algorithm to generate the input text file.4) Run the answer interface generator with the text file to make the HTML/CSS/JavaScript files for the new instance.5) Upload the generated files to the web server and inform the URL to the students.6) A student answers to the given instances using a web browser, where the automatic answer marking function will check the correctness of the answers at the browser.7) The student downloads the answer text files at the browser and submits them to the teacher.8) The teacher runs the student answer analyzer with the answer text files and obtains the solving results of the students for each instance.Fig. 6 displays an example of solving results that is obtained by running the student answer analyzer.

IV. HINT FUNCTION FOR FOUR PROBLEMS TYPES
In this section, we present the hint function for GUP, MCP, EFP and PFP of JPLAS.

A. Basic Concept
In the hint function in this paper, any hint must be generated automatically from the input text file using the corresponding generator.The manual generation of hints will increase the load of a teacher.Since the file contains the correct answer for each question, the hint function will show a part of the characters of the answer as a hint.
Then, only the first character is displayed as the hint for GUP, MCP, and EFP, because they are relatively easy.On the other hand, one or more characters with the number of characters will be displayed for PFP, because PFP is much harder than the others.It has been found that the first character is not sufficient as the hint to solve PFP instances.Besides, to prevent a student from using the hint function facilely, it can be used only after the student clicks the answer button five times and keeps answering it for five minutes as the thinking time.

B. Hint Function for GUP, MCP, EFP
The hint function for GUP, MCP, and EFP will display the first character of the correct answer word for each question where the answer of the student is not correct, when he/she requests it by clicking the corresponding button.Besides, it will show the correct answers to the questions in the instance, if the student cannot reach them after a lot of trials and gives up solving it.Fig. 7 displays the hint for one MCP instance.

C. Hint Function for PFP
The hint function for PFP has two steps.Fig. 8 shows the first step and second step hint for the PFP instance in Fig. 5, respectively.The first step will display the first character of the correct answer and the other characters after replaced by * to hide them, so that a student can know the number of characters in the answer.The second step will replace * by the corresponding character in the correct answer randomly with 30% and display them.Approximately, one-third of the answer can be noticed by the student.

V. EVALUATION
In this section, we evaluate the proposed hint function in JPLAS through applications to 38 junior students taking Java programming course in Okayama University.In this application, we selected sample source codes from the course textbook and generated three instances for each of GUP, MCP, EFP, and PFP, and asked the students to solve each problem type within 20min at each of the four classes as quizzes.Any student could use the hint function if necessary.

A. Results of Individual Problem Types
First, we discuss the solution performances of the students for individual problem types.Table I shows the number of students who submitted answer text files, who used the hint function, and the average correct answer rate among all the students and instances.The average rate for the students who did not use or used the hint function are also shown there inside the brackets.It is noted that only one student among 38 did use the hint function for GUP.
This table shows that the hint function gives the higher correct rate except for MCP.It also shows that as the difficulty level increases, more students used the hint function and the correct rate decreased.Particularly, a big gap can be observed between EFP and PFP.One reason may come from the short time of solving PFP instances.Besides, it may be necessary to improve the hint function for PFP, and to provide another exercise problem type that can reduce this gap.They will be in future works.

B. Results of Individual Students
Next, we discuss the solution results of individual students.Fig. 9 shows the average correct answer rate (%) and the number of button clicking for the hint function for GUP, MCP, EFP, and PFP by each student.A total of 33 used the hint function.Here, we note that this number of button clicking is increased by one for each type when the students clicked it, and is increased by two if he/she clicked it at the both steps for PFP.In GUP, MCP and EFP, most of the students achieved over the 90% rate.On the other hand, in PFP, more than half of them could not reach 50%.It can be considered that many students cannot complete solving the PFP instances within this limited time, and may be insufficient in programming knowledge and skills.

C. Results of Students Using Hint Function
Finally, we discuss the solution results of the students who used the hint function.Fig. 10 shows the number of students who used the hint function by the range of the correct answer rate.It indicates that some students reached 100% by using the hint function, but more students resulted in poor results regardless of the hint function.This result may come from the short time of solving the problems as quizzes.In the hint function, a student can use it only after five minutes have passed as the thinking time since he/she accesses it.Thus, the quiz time 20min was too short for them to freely use the function and solve the questions by referring to the given hints.We have to remedy this undesirable situation by adjusting the thinking time from the next application.

VI. HINT FUNCTION FOR CODE COMPLETION PROBLEM
In this section, we present a hint function for code completion problem (CCP) by extending the one for PFP.CCP is another type of exercise problems where the correctness of an answer is checked through string matching.
Our homework assignments of six types of exercise problems in the Java programming course found that the solution result for CCP is the lowest [20].Therefore, the hint function is important for CCP.

A. Code Completion Problem (CCP)
A question in a CCP instance requests to correct and complete each statement in the given source code that has several blank elements and incorrect ones.Fig. 11 illustrates the answer interface for an example CCP instance.The source code prompts a student to enter three numbers, reads them from the console using a BufferedReader object, converts them from strings to integers, calculates their sum, and prints the result to the console.

B. Homework Solution Results
In this Java programming course, we generated a total of 109 instances for GUP, MCP, VTP, EFP, CCP, and PFP, and assigned them to the students as homework.Thus, no time limitation was imposed.Table II shows the number of students who submitted answer files, the average correct answer rate, and the average number of answer submission times for each instance by the students for each problem type.This table indicates that CCP has the lowest correct rate and the second highest number of submissions among the six types of problems, and is most difficult for them.Thus, the hint function will be very important for CCP.

C. Hint Function for CCP
The hint function for CCP consists of two steps like PFP.Fig. 12 shows the first step hint and the second step hint for the CCP instance in Fig. 11, respectively.The first step will display the first character of the correct answer for each blanked or incorrect element in the given source code, when he/she clicks the corresponding button to request it.The second step will display the whole statement except the blanked or incorrect element where only the first character is correctly shown, and the remaining characters are replaced by *.Then, a student can determine the position of the blanked or incorrect element in the statement as well as the number of characters.

VII. CONCLUSION
This paper proposed the hint function for the grammar-concept understanding problem (GUP), the mistake correction problem (MCP), the element fill-in-blank problem (EFP), and the phrase fill-in-blank problem (PFP) in Java programming learning assistance system (JPLAS).It will display the first or more characters of each correct answer word in the JPLAS answer interface.Besides, the use of this function by a student can be traced by a teacher to analyze the performance and problem difficulty.For evaluations, three instances were generated for each of GUP, MCP, EFP, and PFP, and were assigned to junior students taking the Java programming course in Okayama University, Japan.Their answer results confirmed the effectiveness in helping students solve exercise problems in JPLAS, but the thinking time needs to be adjusted for PFP.In future works, we will evaluate the hint function for CCP, provide diverse hint functions depending on students' levels, generate new instances on hard topics for students, and assign them in Java programming courses.

CONFLICT OF INTEREST
The authors declare no conflict of interest.AUTHOR CONTRIBUTIONS Y. Jing conducted the research and wrote the paper.N. Funabiki and K. Ueda reviewed and finalized the paper.S. T. Aung, X. Lu, and H. H. S. Kyaw collected and analyzed the data.All the authors had approved the final version.

Fig. 10 .
Fig. 10.The number of students by using hints and their correct answer rate.