Intelligent Chatbot for Admission in Higher Education

 Abstract —Universities assist prospective students by giving counselling services and information, which is usually done directly at the educational institution, over the phone, or through social media accounts that is available on the college website. However, increasing visitors cause longer wait times due to officers’ restricted availability, resulting in lower satisfaction among prospective new students. Furthermore, this service is only available during university or college business hours. One possible solution to solve this problem is to use a chatbot to answer frequent questions raised by prospective new students. Intelligent chatbot for university admission is discussed in this study. Botsify platform was used to develop the chatbot for admission. 42 postgraduate students clicked on the chatbot link, most of them from Princess Nourah University. These students interacted with the chatbot by asking numerous questions and receiving responses from the chatbot. The chatbot performance was measured by using both the chatbot usability questionnaire and Confusion Matrix. According to the study’s findings, the chatbot system that was created could correctly and adequately answer questions provided by prospective students while the questions were stored in the chatbot knowledge. The measurement of chatbot performance was done by using Chatbot Usability Questionnaire (CUQ) which consisted of sixteen balanced questions about various elements of chatbot usability. The participants’ number of the Chatbot Usability questionnaire were 22. The result of the CUQ average was 76.6. At the same time, the Confusion Matrix was used to evaluate the chatbot performance and the accuracy of the chatbot was 91%.


tasks.
The oldest chatbot was called ELIZA in 1966, which represents the natural language conversion between human and machine The chatbot also implements a simple or advanced level of artificial intelligence [2].Thus, the chatbot represents a human-computer interaction model and an artificial intelligence program that communicates with users.It communicates in human language by text or oral speech with humans or with other chatbots, and it does so by using Natural Language Processing (NLP) and sentiment analysis techniques.Chatbots are also referred to as smart bots, interactive agents, artificial conversation entities, and digital assistants, among other terms [2].
Artificial intelligence chatbot is unlike traditional rule-based chatbot.It can understand and respond to the humans with whom they are interacting because of advances in natural language processing, natural language understanding, as well as natural language generation.AI chatbots apply machine learning algorithms that allow bots to become smarter over time [3].Besides being useful for imitating human interaction and entertaining people, chatbots are also useful in many fields such as healthcare, education, e-commerce, business, and entertainment.Regarding chatbot users, productivity is the most important motivator, although other factors to consider include social factors, entertainment, and novel interaction opportunities [4].
One of the most effective artificial intelligence techniques is the Chatbot system; it is used to support teaching and learning is the Chatbot system [4].In higher education, most students have access to a smartphone, which means they use internet-based applications frequently.Chatbot systems became increasingly popular for assisting in learning.In real-time, chatbots can provide students with standardized information such as course contents, practice questions and answers, evaluation criteria and deadlines for assignments, as well as advice, campus path directions, and study materials, among other things.These systems not only have the potential to support students and improve their engagement, but they can also significantly reduce the administrative workload of lecturers, permitting them to devote more time to curriculum development and research.
Thus, chatbot technology has the potential to provide students with an engaging learning environment and a more personalized [5].
The use of education Chatbots could improve students' learning experience and assist faculty members through bringing automation into the classroom environment.Chatbots in educational settings improve connectivity, efficiency, and predictability in interactions [6].A chatbot can also be used in educational systems like technical institutions to facilitate communication.It is used to improve student interaction and collaboration, and it has the potential to be a game-changer in today's technologically advanced world.This personalized learning environment, intelligent feedback, a virtual assistant, efficient teaching, and immediate assistance to the students are all provided by chatbots [7].
The problem with admission is that the students need to directly communicate with the university to answer their questions.One of the most prominent problems faced by students seeking access to graduate programs is the ambiguity in obtaining an answer regarding some inquiries related to the admission and registration process to the educational institution.Students take a lot of time searching for valuable and accurate information through universities and colleges' websites.This leads the student to get bored as he navigates between the lists on the site and the links that transfer him from page to page looking for this information.These inquiries could be about the date of registration, the available programs for study, the required tests, and so on.Even if the student decides to go to the university to get answers to their inquiries, this requires time and effort in addition to the various procedures necessary to enter the university campus.It will require the students a lot of action to determine who is responsible for answering the inquiries.Hence, there is a need for this chatbot that shortens a lot of effort and time.Moreover, it can serve more than one student simultaneously and provide trusted and reliable information.This study developed an intelligent system where students can get information, they need to learn about the admission system in higher education through Chatbot technology.
The remaining sections of this paper are organized as follows: Section II describes related work on existing chatbot applications and previous chatbots in education.Section III presents the research methodology and chatbot's evaluation procedures.Section IV presents data collection and analysis and chatbot implementation.Section V focuses on the results of measuring the accuracy of the chatbot, discussing these results and comparing the developed chatbot with others.Section VI summarizes the research paper.

II. RELATED WORK
The majority of Chatbot systems are focused on educational purposes, such as answering students' questions, teaching students how to understand Computer Programming concepts, and assessing students' performance abilities, and providing administrative services [8].Smutny and Schreiberova [9] proposed the first review of the educational chatbots on Facebook Messenger.They examined 89 chatbots and evaluated 47 educational chatbots.The study results illustrated that chatbots are still in their early stages, especially for becoming artificial intelligence teaching assistants.One of the recent intelligent educational chatbot was introduced by Nguyen et al., 2022 in Vietnam.The chatbot was developed to learn two courses: (1) introduction to Programming and (2) Object-oriented Programming.Machine learning and natural language understanding were used for classifying intents and information retrieval.Nguyen And Tran et al. integrated multiple knowledge domains (Integ-Rela model) to manage the knowledge base of the chatbot [10].The results showed that the chatbot helped students with many issues such as the meaning of definitions and finding out the content of lessons.
Nwankwo used Natural Language Processing (NLP) algorithms and Artificial Intelligence (AI) to develop the chatbot system [11].It provided answers to questions pertaining to the admission, examination cell, academics, student attendance, placement cell, grade point average, and a variety of other activities.Academic excellence in educational institutions has been improved using an Interactive Advisory System that includes bots.Accordingly, the work attempted to achieve a sense of balance by proposing an automated -AdvisorBot‖ that was faceless and operated using a bot framework.The design was based on a virtual support system model that was implemented to improve the efficiency of student support and course advice services.In this design, an agent-oriented approach is combined with an object-oriented approach.The result was an implementation-ready specification to effectively support students throughout their academic careers.The system streamlines the advising process through providing easy and quick access to valuable information and giving critical feedback on a variety of issues involved in student advice that would otherwise take a significant amount of time.
Ranoliya and Raghuwanshi et al. created the university chatbot based on the frequency of Frequently Asked Questions (FAQs) to address the problem of completing many tasks simultaneously [12].This chatbot was based on a dataset of FAQs using Latent Semantic Analysis (LSA) and Artificial Intelligence Markup Language (AIML) to provide an efficient and accurate answer to any query.AIML was for responding to template-based welcomes, greetings, and general questions, while LSA was for responding to service-based questions at any time.Any university can use this chatbot to provide interactive answers to FAQs from curious students.Artificial Intelligent Chatbot refers to a virtual assistant system based on artificial intelligence and can answer any college-related question.This system interacts with a virtual human, and it is intended solely for college-level students to use.This virtual machine responds to the students' questions about college-related issues.If an answer is invalid during responding, the system will notify the user that the answer is invalid and then delete it or modify it via the system administrator.Thus, the students will obtain information they need without visiting the college or standing in a long line.
Adams and Raja et al. presented a cognitive virtual admissions counsellor for the master's degree in data science at Southern Methodist University [13].Students could use this chatbot to answer their questions in a very simple and sensible manner.Essentially, the virtual admissions counsellor system provided prospective students with accurate information according to their requirements.After evaluating several technologies, Amazon's LEX was applied for the virtual counsellor chatbot's interaction with users.Student surveys were used to gather and generate training data for the natural language capability, which was then used to deploy the capability.It is currently possible to provide an end-to-end conversational dialogue to solve three categories of questions, which are referred to as -intents‖ on the cognitive virtual admissions counsellor platform.Three categories and potential intents were identified from the survey, including: (1) general academics, (2) financial information, and (3) admissions.These intents allowed the chatbot to determine whether the user inquired about class size, tuition amounts, or semester start dates and accordingly International Journal of Information and Education Technology, Vol. 13, No. 9, September 2023 respond appropriately.Using the virtual admissions counsellor software supported three different intents with accuracy metrics higher than 90%.With LEX as the core service, the virtual admissions counsellor provided successful resolution of potential student inquiries.
Chandraa and Suyantoa developed an Indonesian chatbot for university admission [14].This chatbot was based on a Question Answering (QA) system-a dialogue system to handle chat like a human.The developed system consisted of two stages: training and testing.The training set is a pair of input and target sentences fed into the Seq2Seq model.The pre-processing of the sentences included removing punctuation, lowercasing, and tokenizing.After that, the model was trained based on the Seq2Seq with and without an attention mechanism using a learning rate of 0.001 and three different numbers of neurons of 100, 200, and 300.The trained model was then evaluated in the testing stage.The collected dataset was from WhatsApp conversations about admission to the university.A successful Indonesian QA system based on the Seq2Seq technique has been built.The typical Seq2Seq model without an attention mechanism gets a BiLingual Evaluation Understudy (BLEU) Score of 43.61 when tested on a small dataset from Telkom University Admissions.The model was improved by both attention mechanisms and reversed sentences, yielding a BLEU Score of 44.68, which was accomplished with a bidirectional LSTM encoder that incorporates forward and backward layers and 300 neurons were used.
Gbenga and Suyanto developed a real-time model for responding to admission-related inquiries [15].The study defined a chatbot as a computer-generated application that can have a virtual conversation with a human without making them feel like they are conversing with a machine.The chatbot was a conversational agent using IBM's Watson-an AI intelligence platform to react to admissions-related questions at the university.The user was given access to a messaging platform, an interface where the user can speak with the chatbot through any device connected to the network.The user sends a message to the chatbot, which converts it into plain text and sends it to the natural language understanding component, which extracts its intent and entities before passing it to the natural language processing component.The information sources could be APIs, knowledge bases, data storage, or human intervention to search for the relevant data supplied to the message generator, so, the generated response was transmitted to the chatbot to subsequently respond to the user.The model was created to have a chatbot react to university admissions-related questions using IBM Watson's artificially intelligent platform.Intents, entities, dialogue analytics, and other platform components were necessary for the development of a successful chatbot.The developed system was tested and evaluated using Botium, which used test cases called BotiumScript.The test results demonstrated an accuracy of 95.9% in instances of 212 successful test cases and 9 failed test cases.The chatbot achieved the goal of assisting prospective students and parents with university admission inquiries in a timely, trustworthy, and effective manner.
Nguyen and Le et al. developed National Economics University (NEU) chatbot in Vietnam to support prospective students and parents answering their admission inquiries [16].
The authors introduced an artificial intelligence-based chatbot to help students get daily updates of curriculum, tuition fees, scholarships, admission for new students, and more.Rasa platform in Natural Language Understanding (NLU) was used for developing the chatbot.The platform analyzed the information given by the user to the chatbot.This information contained the intents and Rasa NLU extracted the entities.The deep learning models were applied to help the chatbot decide which actions should be taken at each stage in a conversation.A survey on facebook was conducted to measure the users' satisfaction and the results showed that NEU-chatbot achieved 97.1% accuracy of the testing set.However, a manual update each academic year is needed with new intents and new information as well as handling the misunderstanding of intents.Nguyen and Tran et al. [17] also developed an intelligent chatbot based on ontology technology, which required a knowledge base for supporting users better.This intelligent chatbot was for learning knowledge in the introduction to the programming course.The user communicated with the chatbot to search for content and retrieve knowledge, like definitions, exercises and examples, exercises.This chatbot communicates with students in Vietnamese and provides the students with explanations that meet their requirements.
On the other hand, Smutny and Schreiberova examined 89 chatbots for Facebook Messanger that focused on the education category [18].This study was the first review of the chatbots in education and they classified them based on subject matter, development platform and conversation language.They found that 89% of the chatbots used English language.Results also demonstrated that 46% of the chatbots lacked any discussion techniques and the chatbots used automated answers with additional information from outside the chatbot interface.
Another study done by Okonkwo and Ade-Ibijola reviewed the previous studies of chatbots in education [19].The challenges facing the Chatbot implementation was one of the factors that the study involved.These challenges included ethical issues, programming, user attitude, and evaluation issues.There are some ethical concerns such as privacy, trust, and usability, especially in education.Ruane and Birhane et al. suggested that the chatbot functions should be explicitly detailed and users should determine how to interact with the chatbot [20].Programming is a key challenge because it concerns with how the programmed chatbot gives an accurate answer to the user's questions [21].Effective programming enables the chatbot to learn how to provide suitable answers to users [22].In the user attitude challenge, the users' behavioural intentions are influenced by their attitudes toward the use of artificial intelligence in higher education.Thus, students will be hesitant to use chatbot if they have negative perceptions of using the chatbot systems [19].In evaluating chatbot design, Rapp and Curti et al. [23] found that the evaluating effectiveness and utility, and engagement people were insufficient.This study tried to focus on the challenge of evaluating the effectiveness and utility through applying two methods: confusion matrix and chatbot usability questionnaire to measure the usefulness of the chatbot.

III. RESEACH METHODOLOGY
This section discusses machine learning algorithms, pattern matching, natural language processing, and chatbot evaluation.

A. Machine Learning and Chatbots
A chatbot interacts through instant messaging, and replicates the patterns of human interactions in an artificial manner.Machine learning allows computers to learn by themselves without programming.Multiple types of supervised learning algorithms (which runs in case of labelling training) are used in classification, such as neural networks, decision trees, regression, and support vector algorithms.The Naive Bayes algorithm has been used in this research study.
Naive Bayes is one of various text classification methods.The Naive Bayes classification method is simple and effective; thus it is highly efficient and easey of implementation.This Bayesian classification is employed as a probability learning method, and the classification of each feature in the algorithm is independent of the value of other characteristics [24].
The Naive Bayes algorithm aims to categorize text into certain categories so that the chatbot can determine the user's purpose, restricting the range of possible responses.This algorithm should work effectively because intent identification is the most important phase in chatbot discussion.The algorithm depends on the principle of commonality, which states that certain terms should be given more weight in certain categories based on how frequently they appear in that category [25].Using k-fold cross-validation to test this approach is the most straightforward method.This entails training the chatbot with specific inputs and their related classifications, followed by a test set to see how often the chatbot properly classifies each input.Confusion matrix, accuracy, precision, and recall are used to evaluate the algorithm's performance.The fact that the Naive Bayes algorithm employs a -bag-of words‖ method is a flaw.To determine the input class, the algorithm evaluates all words as a whole and selects the most important ones.This means it does not care about the sequence in which words appear.This is a concern, since certain word rearrangements could cause inputs and classes to diverge.Techniques such as n-gram are used to preserve the order of the words to circumvent [26].The Naive Bayes method is adaptive and intelligent, and it fulfils personalized needs.As a result, it is widely employed in commercial applications.
A chatbot represents an interaction between humans and machines to exchange that happens via voice or read messages.The programmed chatbot works independently from a human operator through answering questions using natural language and answering these questions like a real person.These answers that the chatbot comes up with are a combination of predefined scripts and machine learning.The response of the chatbot is based on what it knows when a question is asked.If the conversation brings it to a place where it does not know what to do, then the chatbot will pass the conversation to a human operator or deflect the conversation.

B. Pattern Matches in Chatbots
There are two types of keyword/pattern matching approaches used by chatbots.The first is analogous to the incremental parsing strategy used by the human brain [27], in which an input sentence is processed in sequence word by word from left to right.The keywords are one-word or multiword, but multiword keywords require each word to be connected to the next, making a continuous keyword pattern.The second method is a direct match, in which the input sentence is evaluated for the presence of keywords in it.The entire input sentence is handled as a single variable, where the available keywords in the database are scanned for matches.The main difference between these two techniques is that the first technique is input centred (words from the input sentence are matched against keywords in knowledge-based), while the second technique is keywords centred (keywords in knowledge-based are being matched against an input sentence).In spite of these differences, both groups propose the same matching paradigm, where one keyword is required to elicit the desired response.Chatbots implement pattern matches to group the text.The Artificial Intelligence Markup Language (AIML) is a structured model of these patterns.AIML is derived from an Extensible Markup Language (XML).The following is the most uncomplicated example of a pattern match: <aiml version=-1.0‖encoding=-UTF-8‖?>.

C. Natural Language Processing (NLP)
Natural Language Processing (NLP) combines AI and linguistics and aims to help computers understand human language statements or words.Natural language processing helps users to do their jobs easily and satisfy their need to communicate with computers in natural languages [28].
Natural language processing is classified into two parts: (1) natural language understanding and (2) natural language generation.These two parts represent the task of understanding and generating the text [29].
Natural Language Understanding (NLU) as a science of language includes sentence structure, phonology (sound), International Journal of Information and Education Technology, Vol. 13, No. 9, September 2023 morphology (word formation), syntax, semantics, and Pragmatics (understanding) [29].Chatbots use NLU to derive context from unstructured input of the user in human language and respond based on the current user's purpose [30,31].The three critical issues presented during the NLU process are the user's thought, interpretation, mechanisms, and general knowledge [32].NLU provides intent categorization and entity extraction while considering the context.Entities are defined by the system or by the user.Contexts are strings that store the item to which the user refers [33].The intent classification model is a classifier, such as a linear Support Vector Machine (SVM) algorithm, or a pretrained model developed by manually categorizing gathered text messages from users into intent categories.
Dialogflow is an example of a natural language understanding platform.Dialogflow simply creates and integrates a conversational user interface into your mobile-app, web-app, device, bots, interactive voice response system, and other applications.Dialogflow assesses several types of consumer input, such as text or audio inputs (like from a phone or voice recording) [34].Natural Language Generation (NLG) is the process of producing meaningful sentences, phrases, and paragraphs from an internal representation [29].
There is a relationship between Chatbot, Natural Language Processing and Machine Learning.A chatbot is simply a robot assistant that can be used in conversation to complete tasks.Text messages or voice commands can be used to carry on the chat.The crucial thing is the artificial intelligence that allows software or robots to communicate with humans to perform simple and sophisticated jobs.While machine learning is used to create a variety of chatbot algorithms, Natural Language Processing (NLP) operates as a sensor, detecting and even imitating human speech patterns.When these two key artificial intelligence aspects are combined, the chatbots become more sensitive to human behaviour and emotions in real-time interactions, resulting in a better user experience [30].

D. Chatbot Evaluation
Two assessment methodologies were employed in this research study: (1) Chatbot Usability Questionnaire (CUQ) is used to assess the usability of the chatbot using a questionnaire built for this purpose.(2) Confusion Matrix is a performance tool for evaluating and measuring the accuracy of the chatbot.

E. Chatbot Usability Test
Chatbots are considered more natural and intuitive than traditional techniques of human-computer contact since it is like human-human interaction.If a website or system is poorly designed, this will reduce the usability of the website or the system.For instance, in a web-based system, a poorly designed search function may result in incorrect information being returned to the user.The same could also happen in the chatbot, if the chatbot incorrectly interprets the user's message or misunderstands the user question [35].
According to the study done by Holmes and Moorhead et al., they applied usability surveys for the WeightMentor chatbot that was created for Facebook messenger [35].The surveys included the System Usability Scale (SUS) scores, User Experience Questionnaire (UEQ), and Chatbot Usability Questionnaire (CUQ).System usability scale contained ten validated statements, which included five positive aspects and five negative aspects of the system.Participants scored each question out of five and the final scores were out of 100 [36].The user experience questionnaire assesses the user experience.It was based on six scales: Attractiveness, efficiency, perspicuity, dependability, stimulation, and novelty).However, UEQ scores need to be compared with a benchmark for assessing the extent to which the system meets expectations [37].The chatbot usability questionnaire focused on the chatbot user experience principles provided by the ALMA chatbot test tool [38].It assessed the personality, onboarding, navigation, understanding, response, error handling and intelligence of a chatbot.The CUQ included 16 statements representing positive and negative aspects of the chatbot.The scale of these statements included five ranks from strongly disagree to strongly agree [38].The Holmes and Moorhead et al.'s study [35] concluded that CUQ was more related to chatbots than SUS and UEQ.Thus, this type of questionnaire was used to test the usability of the developed chatbot.CUQ scores will be calculated using the following Eq.( 1): where m = 16 (number of questions), n = individual question score per participant, and scores are based on a scale of 100.This type of questionnaire was used to test the usability of the developed chatbot in this study.

F. Confusion Matrix
The confusion matrix is another type of evaluation to measure the chatbot's performance.Basically, the confusion matrix contains information to compare the results of the classification accomplished by the system with the results of the classification that should be [29].The confusion matrix measures the performance of the classification problem by using four combinations of actual and predicted values.A True Positive (TP) represents that the model predicts the positive class correctly.A True Negative (TN) represents that the model predicts the negative class correctly.A False Positive (FP) happens when the model predicts an observation belonging to a class while it is not in reality.A False Negative (FN) represents that the model predicts the negative class incorrectly.

IV. DATA ANALYSIS AND RESULTS
The section includes data collection, data reprocessing, chatbot design, and chatbot results.

A. Data Collection and Descriptions
The data was related to admission and registration for postgraduate studies at Princess Nourah Bint Abdul Rahman University.This data included the programs available for registration and the general requirements for master's programs and the special conditions of some of these programs, such as Master of Computing (major: Data Science), Master of Science in Biology (major: Environmental Microbiology and Biotechnology), and Master of Science in Business.These specializations have been chosen to present any knowledge connected to them, for instance, the study plans of the programs and the fees charged.Moreover, among the data that we were keen to provide were the tests required to apply for postgraduate programs and their validity dates.Therefore, this data was available on the official website of Princess Nourah University, Department of Graduate Studies.In addition, an interview was also conducted with one of the officials responsible for admission and registration in postgraduate studies to determine the most common and frequently asked questions that students repeat upon registration.Then the answers to them were noted for use in feeding the chatbot.

B. Data Pre-processing and Chatbot Design
First, gathering the data, the website of the Princess Nourah University was scanned for admission to feed the chatbot and create a datastore for reacting to students' inquiries.Most of the relevant information was collected from the Department of Graduate Studies on the website.The data was arranged as follows: general entrance requirements, admission requirements for each program, and examinations required to apply for graduate programs.Data was also compiled on the study's fees, which involved financial costs based on each specialty.There is also information about the available programs and the study plans for these programs.The investigation and collected data were focused on three available programs: Master's Computing (major: Data Science), Master of Science in Biology (major: Environmental Microbiology and Biotechnology), and Master of Science in Business.
The second approach for gathering data was an interview with one of the faculty in charge of admissions and registration to obtain answers to the most often asked questions that students have during the graduate program application process.From the interview, other questions that the students frequently ask were obtained to store them in the chatbot's data store.
Fig. 1 illustrates the architecture of the chatbot, the data stored in the chatbot in the form of keywords or phrases as shown in this figure.Keyword matching: The keyword matching algorithm will look for keywords in a sentence.An answer will be retrieved if one or more keywords are detected in the user's input text.Table I represents an example of the keyword to understand better how the keyword matching algorithm works.It represents instances from the chatbot.The first row shows the chatbot retrieved when it recognizes one keyword.The algorithm recognizes the keywords in the second example and extracts the answer.
Phrase-based: The phrase-based algorithm compares the chatbot's prestored queries with the user's inquiry and discovers the best match in the chatbot datastore.Most frequently asked questions were saved as prestored questions, so that the student's query was compared with this previously stored phrase, and the chatbot displays the relevant response to the user.Table II displays the chatbot's example of the phrase-based algorithm.

C. Chatbot Implementation
The Botsify platform was utilized to create the Chatbot.Botsify is a machine learning chatbot building tool.It can manage an unlimited number of conversations at once, making it ideal for recurring questions, and it stands out for its quick response time and low cost of service.Furthermore, it supports the Arabic language and gives a comprehensive analysis and status report on the Chatbot's performance.Bot learning, conversational forms, chatbot training, media blocks, datastore, and stories are all available to Botsify.Botsify takes advantage of NLP, full-text search, and Google Dialogflow.Behind the scenes, there are Botsify algorithm analysis and responses to conditions, keywords, phrases, punctuation, styles, and word counts.In addition, the system has hundreds of checkpoints in place to ensure that messages are matched with appropriate responses.To check for similarity, Botsify uses the Naive Bayes machine learning technique.It seeks to show the probabilistic relationship between distinct variables and identify which group they are most likely to belong to while taking the variables into account.Fig. 2 shows the welcome message of the chatbot and Fig. 3 shows question and answer about the available Master programs.One of the benefits of this platform is that the chatbot is trained using a platform feature that records all inputs submitted by the user that were not recognized by the chatbot.The platform also provides a general report on the chatbot and the conversations that were conducted with all users to provide an opportunity to improve the chatbot's performance.

D. Chatbot Results
To measure the effectiveness of the developed chatbot, the link of the chatbot was visited by 42 postgraduate students.Most of them were from Princess Nourah University.These 42 students asked multiple questions and received responses.The most important questions were about the tuition prices for the programs and the admission requirements.22 chatbot users completed the Chatbot Usability Questionnaire (CUQ).The CUQ included sixteen balanced questions about various elements of chatbot usability.These 16 questions focused on the personality (Questions 1 to 4), onboarding (Questions 5 to 6), navigation (Questions 7 to 8), understanding (Questions 9 to 10), response (Questions 11 to 12), error handling (Questions 13 to 14), and intelligence of a chatbot (Questions 15 to 16) to assess the chatbot.Eight of them are related to the positive features of chatbot usage, which are Questions 1, 3, 5, 7, 9, 11, 13, 15, respectively, and the remaining eight are related to the negative aspects.

E. Analysis Questionnaire Results
The analysis of the response of each question in the CUQ is presented in Table III.These questions were ranked into strongly disagree (S.D.), disagree (D), natural (N), agree (A) and strongly agree (S.A.).The results showed that over 90% of students found that (1) the chatbot's personality was realistic and engaging, (2) it explained its scope and purpose well, (3) it was easy to navigate, and (4) its responses were useful, appropriate, and informative.Over 95% of students found the chatbot easy to use and 100% of them did not find it very complex.86% of the participants found the chatbot was welcoming during the initial setup and they realized that the chatbot understood them well.Moreover, around 73 % of students found the chatbot too robotic and it recognized a lot of their inputs.
Table V illustrates the average score (mean) for each question and the standard deviation.The results in this show that the positive (in essence, odd-numbered) questions generally have high mean scores greater than 3.0; this demonstrates that the chatbot user experience was generally positive.For the developed chatbot, the lowest scoring positive question was Q13, and it is stated -The chatbot coped well with errors or mistakes.‖The highest were Q3, Q5, and Q15, and these questions were -The chatbot was welcomed during the initial setup,‖ -The chatbot explained its scope and purpose well,‖ and -The chatbot was very easy to use‖ respectively.This demonstrated that, on average, the participants found the chatbot welcoming, explained its scope and purpose, and easy to use, but it did not cope well with errors.On the other hand, the lowest scoring negative questions Q6 and Q16 for the developed chatbot were -the chatbot gave no indication of its purpose‖ and -the chatbot was very complex‖ respectively.This signified that, on average, participants found that the chatbot gave an indication of its purpose and it was not complex.The highest score was Q2, -the chatbot seemed too robotic‖, which means the users felt that the chatbot appeared automated.The SD (standard deviation) is the variation or deviation from the mean.Most of the questions have a Standard Deviation less than 1.Despite Q2, Q4, and Q8 having very high SD (1.0 or greater), which means that there was much variation between the answers to these questions.

F. Measuring the Accuracy of Chatbot
To determine how accurately the chatbot responded to the False Positive (FP): The chatbot can handle the unexpected user request or message correctly.
True Negative (TN): The chatbot can handle the expected user request or message incorrectly.
False Negative (FN): The chatbot can handle the unexpected user request or message incorrectly.
The accuracy, precision, recall, and F-Score of the chatbot were calculated.The sample size was 42 users that had real interaction with the chatbot, and the measurements were calculated based on this sample size.According to the results, the precision rate is 96.59%, which represents the correct prediction of intent.The recall rate is 94.66%, which represents the number of intents that are correctly predicted.The F1-Score rate (harmonic mean) is 95%, and the accuracy rate is 91.97% meaning that the developed chatbot is working well.This research study used both the chatbot usability questionnaire and the confusion matrix to provide various measurements that can tell us the performance of the chatbot.
Comparing the results of previous research that created chatbot for admission with the results of this study is shown in Table VII.The process for evaluating the developed chatbot was based on confusion matrix and chatbot usability questionnaire to measure the effectiveness of utilizing the chatbot, while other studies used confusion matrix to measure the accuracy of the chatbot.This study applied chatbot usability test to assess items that are closely related to the chatbot.These items included personality, onboarding, navigation, understanding, response, error handling, and intelligence of the chatbot.The results demonstrated that the chatbot user experience was generally positive.Adams et al. [13] Three intents were chosen to demonstrate the process and the capabilities of CVAC (Cognitive Virtual Admissions Counsellor).Each of these intents was trained and tested with cross-validation, and all provided accuracy scores greater than 90%.
Nguyen et al. [16] A survey on facebook was conducted to measure the users' satisfaction and the results showed that NEU-chatbot achieved 97.1% accuracy of the testing set.
Gbenga et al. [15] The test result gave an accuracy of 95.9%, with the instance of 212 successful test cases and 9 failed test cases.
V. CONCLUSION This study aims to save time and effort for the administrative staff and facilitate access to information for students and parents on admission inquiries in a timely, efficient, and reliable manner.To achieve this aim, the chatbot was developed for admission into higher education at Prince Norah University.With this solution, the chatbot can respond to basic information, thus reducing the number of calls, emails, and saving time in higher education.Botsify platform was used to develop the chatbot and it was tested by postgraduate students.Around 42 of them interacted with the chatbot by asking numerous questions and receiving responses from the chatbot.The chatbot usability questionnaire and confusion matrix were used to measure the performance of the chatbot.According to the study's findings, the accuracy of the admission chatbot was 91.97% and the F-Score was 95%, which means that the developed chatbot was correctly and adequately answering students' questions.Therefore, the chatbot delivered answers that were considered efficient and accurate responses, which helped students to locate appropriate information quickly instead of reaching the administrative officers to ask them.However, the datastore's content in the chatbot must be manually updated each year by staff with fresh data and records to cope with the new academic year information.The chatbot's also needed to train thoroughly over time to improve the Chatbot's performance.For future work, the admission chatbot could be expanded by including more data to answer of the user's inquires or questions.Version two of the admission chatbot with voice assistant can be developed the Google Assistant platform to save time for typing.Moreover, addressing the user attitude challenge and the negative perceptions of students toward chatbot applications <category> <pattern> ‫او‬ ‫عهًٍه‬ ‫حىصٍاث‬ ‫او‬ ‫عهًٍه؟‬ ‫حىصٍاث‬ ‫عهى‬ ‫أحصم‬ ‫كٍف‬ ‫انخىصٍاث‬ </pattern> <pattern> How do I get scientific recommendations?Or scientific recommendations or recommendations <pattern > <template> ‫حخزجج‬ ‫انذي‬ ‫انقسى‬ ‫ين‬ ‫انخىصٍاث‬ ‫عهى‬ ‫انطانبت‬ ‫ححصم‬ ‫/<ينه‬template> <template> The student gets recommendations from the department she graduated she graduated from <template> </category> </aiml> The machine gives the output as follows: Human: ‫عهًٍه؟‬ ‫حىصٍاث‬ ‫عهى‬ ‫أحصم‬ ‫كٍف‬ (Human: How do I get scientific recommendation?)Chatbot: ‫ينه‬ ‫حخزجج‬ ‫انذي‬ ‫انقسى‬ ‫ين‬ ‫انخىصٍاث‬ ‫عهى‬ ‫انطانبت‬ ‫ححصم‬ (Chatbot: The student gets recommendations from the department she graduated from)
Score Median ScoreInternational Journal of Information and Education Technology, Vol. 13, No. 9, September 2023 user's inquiries, the confusion Matrix was applied the chatbot.True Positive (TP): The chatbot can handle the expected user request or message correctly.

TABLE IV :
CUQ SCORE IN ASCENDING ORDER

TABLE V
Table VI presents the admission chatbot values of TP, TN, FP, and FN.

TABLE VII :
COMPARING THE RESULTS OF DEVELOPED AND SIMILAR CHATBOT SYSTEMS Results The developed chatbot The mean CUQ scores were 76.6 with standard deviation of 11.5, which means the chatbot user experience was generally positive.The results were as follows: The Precision rate is 96.59 %.The recall rate is 94.66%.The F1-Score rate is 95%.The accuracy rate is 91.97%, meaning that the developed chatbot is working well.