Exploring Students ’ Feedback in Online Assessment System Using Opinion Mining Technique

Opinion mining has been widely used in recent online reviews or feedback due to its ability to analyse text-based data. The use of this technique for analysing data from students’ feedback needs to be addressed, since most educational institutions are focusing more on questionnaires based on the Likert-scale rather than on the open-review type. To this end, there is a lack of online assessment systems that could automatically analyse open-review questionnaires. Therefore, the main aim of this study is to analyse students’ feedback in an online assessment system through the opinion mining technique, by focusing on textual form data derived from the open-review questionnaires. To achieve this aim, an opinion mining feedback system, known as the OMFeedback, was developed. The Vader Sentiment Intensity Analyser was adapted for processing students’ feedback and the lexicon based approach was used for analysing the words. In addition, the OMFeedback incorporates the capitalisation of words and emoji features to enrich the capability of the system. This newly developed system could lead to new paradigms in educational institutions for enhancing students’ learning process and for guiding them through their learning journey.


I. INTRODUCTION
Analysing students' feedback in online assessment systems is a not new concept; educational institutions use this feedback for educational insight. The purpose of obtaining such feedback is primarily to improve the teaching quality and to help the academic administration better understand students' perspective on the teaching and learning process [1]. Online assessment systems would usually consist of Likert-scale and open-review questionnaires. The Likert-scale comprises of a set of questions, such as multiple choice or scale rating, while the open-review has textual feedback questions that ask for opinions or comments. The set of questions in the Likert-scale would typically cater various dimensions, such as a lecturer's teaching style, presentation skills, time management, and course materials. Meanwhile, the textual feedback provides students with the opportunity to point out certain issues that are not directly covered in the Likert-scale questions. Balahadia et al. [2] asserted that most educational institutions rely on Likert-scale questions for easy analysis, compared to the textual feedback, which is hard to digest because of the high Manuscript  amount of text data [3], [4].
The aforementioned students' feedback would generally be available in an unstructured format, which would require extra analysing effort to gain fruitful conclusions from these feedbacks. Advanced data mining techniques that are capable of exploring the unstructured patterns inside the textual data are required. One of the most active techniques for understanding various types of students' feedback is the opinion mining technique. It is an area of research within sentiment analysis that could examine human's opinions, comments, feelings or behaviours towards various entities, such as services, products, organisations, individuals, events, issues, and highlighted topics [5]. According to Ravi and Ravi [6], opinion mining and sentiment analysis are interrelated areas of research, which extract texts or words that have been commented in numerous levels. Sivakumar and Reddy [7] reported that these texts can be classified as document level, sentence level, and word level. These different levels have text polarity that can be determined by checking the occurrences of a positive, negative or neutral word. For instance, if a positive word occurs more frequently than a negative word in a sentence, then the sentence is deemed a positive sentence.
Basically, the opinion mining technique can be divided into machine learning and lexicon based approach. The main objective of these techniques is to determine whether the written comments contain positive, negative or neutral opinions. Positive opinions represent the necessary things, negative opinions means unnecessary things, and neutral opinions are used for a better distinction between positive and negative words. Examples of positive words would include "joy", and "trust"; negative words would be "anger", and "fear"; and neutral words are "but", and "surprise" [8]. Machine learning focuses on building models with the aid of large training datasets to determine the features of text orientation [5]. Whereas, the lexicon based approach uses words contained in the sentiment dictionary (e.g., Vader Lexicon, SentiWordNet, and General Inquirer) and matches them with the collected data to determine the polarity of words [9].
Numerous machine learning techniques have been used to analyse students' feedback, such as Naïve Bayes (NB), Support Vector Machine (SVM), neural network, and k-nearest neighbour. Dhanalakshmi et al. [10] highlighted that NB is the most commonly used method to calculate the possibility of a given text belonging to a particular feature; SVM works best for classifying sparse text data; neural network employs multiple layers of neurons for text classification; and k-nearest neighbour uses Euclidean distance to calculate the similarity of text data. Several Exploring Students' Feedback in Online Assessment System Using Opinion Mining Technique studies claimed that neural network is the perfect technique for opinion mining [2], [10]- [12]. However, a study by Hutto and Gilbert [13] found that most machine learning techniques have drawbacks. First, these techniques often require large training datasets to represent various features. Second, the techniques are often computationally expensive since they need high processing time and large memory spaces. And third, the features extracted from the text are not easily interpretable and are therefore, more complex to modify, extend or generalise.
Recently, several studies revealed that the lexicon based is an optimal approach since it neither needs a large number of text data nor high computation power to produce accurate results [1], [4], [9]. This is in accordance with the results obtained by Hutto and Gilbert [13], who found that such approach can generate a high accuracy value. Previous studies on opinion mining have also shown that the capitalisation of words and the use of emoji characters may emphasise a user's intent and enhance the expressivity of the feedback [13], [14]. However, only a small number of studies have utilised these features to enrich the ability of the opinion mining technique to analyse students' feedback. Based on these reasons, this study has developed a new system to analyse students' feedback, known hereafter as the OMFeedback system. This system uses the lexicon based approach and incorporates the capitalisation of words and emoji characters that are actively used in most online-based systems.

II. METHODOLOGY
To design the OMFeedback system, this study uses sequence diagrams as shown in Fig. 1 and Fig. 2. This system consisted of students and admin as the main users. Before these users can use the system, they need to log in to the system using their username and password. Then, students need to select the name of the lecturer from a drop-down list and write their feedback regarding the teaching and learning process. The lexicon based approach, which are underpinned by Vader Sentiment Intensity Analyser (VSIA) would be used to examine and calculating the polarity of each written word in the feedback and appear the opinion results as positive, neutral and negative scores. Eventually, these feedbacks and opinion results can only be accessed by admin for further actions.   3 exhibits the process of opinion mining in the OMFeedback system. This figure consists of input, process, and output phases. Initially, students' feedback is taken as inputs for the subsequent process. The VSIA will perform two sub-processes, namely, keyword identification and calculation of total scores.  In the keyword identification process, as shown in Fig. 3, each word in the feedback are compared with the words stored in the Vader Lexicon database. This database consists of a collection of words that have fixed polarity of scores. The polarity of scores is then assigned to each word and the total scores of these words are calculated using the lexicon based approach. The OMFeedback system will display a result that contains the polarity of positive, negative, and neutral scores for all of the students' feedback. Table I exhibits some examples of polarity, words, and their scores that are stored in the Vader Lexicon database.
This study used WxPython and Python 3.7 to develop the interface and the interaction of the OMFeedback system. Table II shows the algorithm of the main process in the system. First, students need to choose the name of the lecturer from a list and write down their feedback in the space provided. Then, the compiler will read these input and the VSIA will analyse the feedback text, and then assigns positive, negative, and neutral scores. The lecturer's name, the feedback text, and the score will then be stored in the International Journal of Information and Education Technology, Vol. 10, No. 9, September 2020 Vader Lexicon database. Finally, the sentiment scores (positive, negative, and neutral), will be displayed to the administrator in the form of pie charts. Table III shows the algorithm used by the administrator to view the results of the OMFeedback system.

III. EXPERIMENT AND RESULTS
This study tested the effectiveness of the OMFeedback system using feedback from third-year undergraduate students from the Department of Computer Science, National Defence University of Malaysia during the second semester for the 2018/2019 session. In this phase, 18 students were tested as a sample because they have the same lecturer. First, the students were asked to choose the name of the lecturer, class group, course name, and course code. Next, they were asked to write their opinion or comment in the feedback space provided, as shown in Fig. 4. Once a student has submitted the feedback form, VSIA will identify the keywords and assign the specified values obtaining from the Vader Lexicon. Then, this lexicon based approach will calculate the value of scores and display the results, as shown in Fig. 5. Based on this figure, all students' feedback has different polarity scores for being positive, negative, and neutral. The last two students have interesting scores as they use the new features of capitalised words and emoji characters to better express their feelings and opinions. For example, when a student wrote "very BAD lecturer :(", the lexicon based approach yielded a high negative score (79.4%) since both features simultaneously in a sentence.  The above results can be summarised in the form of a pie chart that can offer more comprehension about the students' International Journal of Information and Education Technology, Vol. 10, No. 9, September 2020 feedback. The pie chart in Fig. 6 shows three coloured sections. The yellow, pink, and green colours represent positive, negative, and neutral opinions, respectively. As shown in the figure, the proportion of students who made neutral opinions exceeds the sum of those who made both positive and negative opinions. In other words, most students have neutral opinions regarding their lecturer's teaching (61.3%). Meanwhile, several students expressed positive opinions (34.9%) and only a small number of students' feedback indicated negative opinions (4.7%).

IV. CONCLUSION
The present study was designed to explore students' feedback using opinion mining technique by integrating the lexicon based approach in the OMFeedback system. This system is capable of accepting textual opinions from students. By integrating the lexicon based approach, the OMFeefback system can automatically analyse students' opinion about their lecturer's teaching skills at the end of every semester. One of the most surprising findings to emerge from this study was the score given by the simultaneous use of capitalised words and emoji features in a sentence, which revealed a significant value. This value corroborates the ideas presented by Gkontzis et al. [15], who suggested that an analytical system with values that are close to 70% is considered as efficient. Thus, the 79.4% obtained from the new features of capitalised words and emoji characters in the students' feedback can be considered as efficient too.
The current study was limited in several ways. First, the OMFeedback system used a Vader Lexicon database, which has a limited number of words. The database does not store words that are related to the educational context, such as "late", "discipline", and "marks". Second, the OMFeedback system was unable to detect spelling errors and assigned a neutral score to misspelled words, which eventually led to inaccurate results. The final limitation was that the system had only focused on students' feedback that was written in English because all the words in the Vader Lexicon database are stored as English. Based on these limitations, future studies should include more words that relate to the educational context in the Vader Lexicon database. To achieve better results, it is essential to integrate a spelling and grammar checker in the OMFeedback system. Finally, future studies are recommended to import a translation programme in the OMFeedback to allow students who are not proficient in the English language the chance to use this system. Overall, educational institutions are encouraged to facilitate this new initiative to increase the effectiveness of their online assessment system.

CONFLICT OF INTEREST
The authors declare no conflict of interest.

AUTHOR CONTRIBUTIONS
Muslihah and Suzaimah led the research; Sharmelen developed the OMFeedback system; Nor Asiakin and Norulzahrah collected the data; Muslihah and Noor Afiza analysed the data; Muslihah wrote the paper; all the authors had agreed and approved the final version of the paper.  expertise are in the area of cyber security, disaster management system, big data analytics, human computer interaction, artificial intelligence & robotics and blockchain technology. Afiza is also a professional technologist, a recognition given by Malaysia Board of Technologist.