Adapting e-Learning for Japanese EFL Students

This study examined the effect e-learning had on second-year Japanese university English as a Foreign Language (EFL) student Test of English for International Communication (TOEIC) scores, which is a standardized English test. The English language learner e-learning system was developed by Gyuto-e ASP and had 800 multiple-choice listening questions. The four research questions were focused on the following: the possible differences in the TOEIC scores between the students who had completed the online module and those who had not; the possible score differences between genders; and whether any students had completed the e-learning but had not had any significant increases in their TOEIC scores. It was found that all tests were statistically non-significant at a 5% level. Further research is to be conducted on improving the e-learning program usage so that it has a more direct and meaningful effect on the students’ test scores.


I. INTRODUCTION
Kyushu Institute of Technology (KIT) recently adopted several e-learning programs, such as ETS's Criterion Online Writing, Gyuto-e ASP for English Learners, and the VI-C course for sophomores. E-learning, which uses information and communication technology (ICT) to formulate, organize, and create learning experiences without any boundaries [1], provides lessons on digital devices, such as computers or supported mobile devices. E-learning has been around for more than twenty years; however, more recently, e-learning has become more learner-oriented, with students being able to control the appearance of the virtual elements and having complete command over their study course [2].
Since the Educational Testing Service (ETS) Criterion contract ended, KIT has been considering English language e-learning. However, the university-wide budget cuts delayed the offer of a new e-learning program for second-year students. As affordable and effective tools were needed to ensure positive English proficiency results, in 2019, one of the authors, Watanabe, conducted a trial run of a Gyuto-e program in two classes to provide basic online listening exercises along with the regular class studies. This trial online course appeared to improve the students' listening skills and revealed the wider scale potential of such courses at the university. In 2020, KIT adopted a Gyuto-e e-learning program for an integrated English course that combined classroom teaching and e-learning. The objective of this article is to discuss the development and present situation of Manuscript  this integrated program at KIT.

A. Historical Development of e-Learning
Where the actual term e-learning arose is a matter of debate, but [3] argued that the term arose during the 1980s; however, computer-assisted language learning has been part of language teaching and learning since the 1960s. Over time, as e-learning became more user-friendly; it was more widely accepted, evolving from basic -computer-based‖ online learning to personalized learning modules. Reference [4] stated that e-learning has benefited from ICT developments that allow educational tools to be adjusted for focused teaching and learning.
The Global Market Analysts reported that E-Learning Market size surpassed USD 250 billion in 2020 and is anticipated to grow at an exponential compound average growth rate (CAGR) of over 21% between 2021 and 2027. The advent of several new technologies, such as AI, VR, and cloud-based Learning Management Systems (LMS), will drive the market growth. The emergence of an AI-enabled e-learning solution will help in the development of smart content, digitized study guides, and real-time questioning [5]. The largest LMS is Moodle, which has 298 million users in 2021, followed by Edomo with over 100 million users (2020 figures) and Canvas (Blackboard) with more than 30 million customers [6]. A wide variety of accessible technology and e-learning devices have been developed to assist with study courses, the number of which increased significantly in 2020 when many classes had to go online. E-learning has been extensively adopted by schools and corporations and adoption is projected to further increase. The only remaining issue is whether there is enough evidence for the effectiveness of these e-learning programs.

B. E-Learning Programs and Trends
E-learning programs and devices can be categorized into synchronous (real-time) or asynchronous (on-demand) learning. On-demand courses generally involve self-paced learning and include reading resources, audio, video, forums, wikis, etc., whereas real-time devices are where both the teacher and student are present at the same time, such as webinars, virtual classrooms, and video meetings. Reference [7] looked at the e-learning factors and determined that e-learning service quality was the most important factor influencing student satisfaction, followed by the e-learning instructor, course material quality, and e-learning administrative and support service quality. Reference [8] also examined e-learning learner-content interactions during COVID 19 and concluded that while the COVID-19 pandemic was a challenge, it was also an opportunity to examine the relationships between learner content and e-learning quality. They aimed to determine the impact of e-learning quality on learner satisfaction in the current context of COVID19, for which a structured questionnaire was conducted on 435 graduate and undergraduate management students, from which statistically significant relationships were found between the e-learning content and the e-learning quality and the e-learning quality and student satisfaction. In short, the e-learning quality was found to have a significant positive relationship with student satisfaction, and this relationship was not found to be affected by the pandemic.
One of the e-learning trends in the past ten years has been blended learning, which is the process of uniting two or more teaching approaches, such as instructional technologies, pedagogical approaches, web-based technologies, and job tasks [9]. Blended learning -combines traditional classroom sessions with e-learning and self-study‖ [10], with the purpose being to produce study media as a collective unit to increase the learning effect [11].
A second trend that has been seen to have great future potential is gamification, which involves the enhancement of e-learning services by integrating gaming elements that both independently and interactively involve learners in business and education segments [12]. The player/learner involvement planning process can be separated into gaming software development and the rational incorporation of tasks. Learning based on games can be especially beneficial for developing skills as it offers both practice and advice in a format that is enjoyable, appealing, and inspiring to learners [13]. Gamification software has been in demand and had a global market value of USD 10.19 million in 2020, with a projected income of USD 38.42 million in 2026, an annual growth rate of 25.10%. 1 It may surprise some educators, but -gamification players did score 14% higher in skill-oriented tests, 11% higher in factual information, and 9% higher in remembering rates, [and] 79% of business personnel and scholars said that the gamification method of learning made them more productive.‖ 2 The third trend in e-learning, particularly when study times are compressed and the content is short and modest, is microlearning; that is, when a curriculum is delivered in parts in a connected and iterative process. The microlearning medium can be either e-learning or traditional and can be used for school or business learning [14]. This has also been referred to as subscription learning, for which learners subscribe to chunks, with the two-way study generally being less than ten minutes. There is a certain learning order and only when the learner finishes a sequence can they receive technical or electronic access to further information, which is referred to as -Integrated Micro-Learning‖ [15].
A fourth trend involves continuous or lifelong learning, which involves an unceasing quest for expertise and knowledge for a personal or occupational objective. -This 1 -Gamification Market-Growth, Trends, COVID-19 Impact, and Forecasts (2021-2026).‖ Available: https://www.mordorintelligence.com/industry-reports/gamification-market 2 -Adoption of Personalized Learning,‖ (February 3, 2016). Available: http://elearninginfographics.com/adoption-personalized-learning-infographi c/ trend can be defined as the extension of educational facilities beyond the orthodox school ages and to aid education as a tool to improve the eminence of life‖ [16]. The three constant learning concepts are equality of opportunity, the centrality of the learner, and high quality and relevance. This learning is usually related to work-based learning for short-term necessity, and life-based learning that can help people gain new employment or fulfill a long-term goal [17].

III. RATIONALE FOR IMPLEMENTATION
Kyushu Institute of Technology is a national university focused on engineering, which means that only a limited number of English classes are offered as most other classes are dedicated to engineering, science, math, and physics. Students are required to take two once-a-week English classes per semester until the end of their sophomore year. The English section offers no English classes during the junior and senior years except for some short-term intensive elective classes during summer or spring breaks.
Nevertheless, since KIT implemented its rigorous Global Engineering course that requires 600 points on the TOEIC, all students are expected to achieve 600 points or more on the TOEIC test. Consequently, some students dropped out of the course because they were unable to fulfill the English language requirement. The situation became so alarming that the English section improved their English classes to help students raise their English proficiency in two years.
The goal seemed almost unobtainable as few English classes are available every week. To achieve the objective within the framework, the regular English classes needed to be intensified to ensure the class content was more effective and affordable. Therefore, it was suggested that a reasonably priced e-learning program combined with regular classroom teaching could maximize the effectiveness of the English program.

IV. BACKGROUND TO THE SOFTWARE
Developed by professors at Hiroshima City University, the -Gyuto-e‖ e-learning program was developed to provide a massive amount of study content to complement the lack of English study hours by Japanese students [18] [19]. Based on this concept, Gyuto-e offers reading, listening, grammar, speaking, and writing from basic to advanced levels. All skill content is intensive with its listening program, for example, providing 800 listening questions at the beginning and the intermediate levels and 720 questions at the advanced level. The listening questions are similar to those in the TOEIC Listening and Writing test, that is, listening audios followed by multiple-choice answers [20].
The user interface is simple and easy to follow (Fig. 1), and the teacher's interface has various simple-to-operate functions available; however, all the buttons and explanations are in Japanese (Fig. 2). The functions include study settings, can flag -cheating‖ behavior by students' answering in 3 seconds, review settings with an adjustable accuracy rate, such as forcing students to review until they attain 80% or more accuracy, and a list of study data that shows the question completion rate, the analysis of each question, the review rate, and questionable study behavior, which is flagged as an -inappropriate‖ answer or cheating (Figs. 3-5).    Compared with other e-learning materials available in Japan, the affordability of Gyuto-e is excellent. For instance, the 800 questions for one listening package costs only 220 yen (tax included) per student, which is equivalent to USD 2 or less. As we have around 500 sophomore students, adopting the listening package would cost only 110,000 yen, which is less than USD 1,000. The cost factor is significant because KIT, which is a national university, must cut its budget by 1% every year due to a government policy. Therefore, cost-effectiveness is the key to e-learning program choice when faced with an ever-shrinking budget and no special funds are available.

V. PROCEDURES
A trial run of Gyuto-e was conducted in the fall semester of 2019, for which the beginning listening package was adapted for a freshmen class to verify that the program was usable and could be integrated effectively into classroom teaching. After the semester was over, it was confirmed that e-learning could be easily blended with classroom teaching and introduced into other English classes on a large scale.
Because Japanese students at KIT lack all English skills, the listening package was chosen as it was surmised that the exercises would help in improving their English proficiency. In addition, the English teachers at KIT empirically know that listening competency can be more easily improved than the other skills. Another reason was that the TOEIC Listening and Reading tests measure listening comprehension; therefore, raising this score could positively affect the students' performances in the Global Engineering course at KIT.
In the spring semester of 2020, around 200 sophomore students were selected for the large-scale Gyuto-e listening program implementation. These students were all -mid-level‖ learners of English at KIT, and were to study the beginning level of listening as well as attend classroom study; five points were awarded to those who completed 400 questions within the first quarter and another five points were awarded to those who completed 400 questions within the second quarter. The students took the TOEIC standardized test in December 2019 and again in December 2020, which was after the e-learning was completed in August 2020. It was expected that the results would show some improvement.

VI. RESEARCH QUESTIONS
The research questions were as follows: 1) Is there a significant difference in the TOEIC scores after completing the e-learning program? 2) Is there a significant difference in the TOEIC Scores between the students who completed the e-learning and the students who did not? 3) Are there any cases of students who completed the e-learning but did not have significant increases in their TOEIC scores? 4) Is there a significant difference between genders in the e-learning percentages and the TOEIC scores?

VII. RESULTS
The pre-scores were the students' TOEIC scores from December 2019 (Table I), and the e-learning data were the question completion rate, with the maximum value being 800. The post-scores were the students' TOEIC scores in December 2020. Of the 192 students who participated in the study (Table II; Fig. 6), 173 completed the e-learning and 19 students failed to complete the program. Of the 173 students who completed the e-learning, 118 students had improved test scores; however, the scores for 47 students went down; and the data for eight students were incomplete because they missed either the pre-or post-tests.
It was found that 80 students had -cheated‖ by answering listening questions within three seconds, meaning that they did not even finish the listening before they answered. This cheating behavior was not necessarily an indicator of the decreased scores in the post-test as only 17 out of the 80 had decreased scores while the others had increased scores. The 17 cheaters had an average 12% cheating rate with the highest percentage being 49% and the lowest being 1%. These 17 students had an average 96-point decline in their TOEIC scores.
Probably because of a lack of clear instructions, only seven students took advantage of the review function in the e-learning program to review 10 or more questions. They reviewed an average of 60 questions, which was far less than the 800 questions the e-learning offered. Nevertheless, their average TOEIC scores improved from 441 to 498. To eliminate outliers, the interquartile range (IQR) was used. First, the third and first quartiles were calculated. Then, from Q3, Q1 was subtracted and the result was multiplied by 1.5. Then, the result of this multiplication was added to the mean and subtracted from the mean. Every value in the data set, which was below mean-1.5*IQR, and above mean+1.5*IQR, was eliminated from the analysis (Table III).  The first research question regarding the e-learning program completion and the possible difference in TOEIC scores was not found to be statistically significant. A closer review of the participants found that there were significantly more male participants (168 out of 192) than female. Female students tend to have better mean results than male students. However, as there was a higher standard error in the female students, the statistical tests showed no significant differences.
To answer the second research question, a simple comparison was made between the students who completed the e-learning and did the post-test and those who did not complete the e-learning program but took the post-test, from which some positive differences were found. However, the data quality was not sufficient to adequately answer the question.
For the third question, which was focused on students who had completed the e-learning but had no significant increases in their TOEIC scores, out of the173 students who completed the e-learning, 118 students had improved test scores and 47 students had lower test scores, which was far more than had been expected. Nonetheless, there was again insufficient data to definitively answer this question.
As for the final research aim about the possible gender differences, the t-tests showed no significance in the pre and post-scores, t(843.16) = -1.7868, p-value = 0.07433, or in the pre-scores t(84.536) = 1.0237, p-value = 0.3089, or the post-scores, t(111.97) = 1.8404, p = 0.06836, though these latter scores were closer to significance.

VIII. DISCUSSION
As the results proved to be non-significant, this revealed International Journal of Information and Education Technology, Vol. 12, No. 4, April 2022 two primary problems: a) the students were doing the e-learning too quickly without taking time to adequately think through each item, and/or b) the students were not using the review mode to identify their mistakes or check the correct answers. Therefore, the English section decided that this year they would award five points for the completion of the e-learning and five points for doing the review mode. In the review function, the e-learning system will be set to require students to raise their score percentages to 80% or above. This may lead to a significant improvement compared to the last year when only seven students took advantage of this function to improve their TOEIC scores.
To further expand on the e-learning, the English section is encouraging students to use the Language Lounge to practice the language and offer TED-like talks in English and use the learned grammatical forms. The Language Lounge is also going to offer website pages for students to introduce themselves and provide interesting information that they would like to share with other students. These two initiatives utilize aspects of Knowledge Space Theory, which recognizes the importance of applying student knowledge to their own needs [21], with a focus on what the students can do (their proficiency) and what the student is ready to learn. This personalized, multifaceted, skilled approach to learning is, in short, another e-learning trend. Reference [22] noted that individually tailored learning provides an occasion for students to participate fully and become co-creators in selecting their own content and organizing their own learning directions.

IX. CONCLUSION
This research investigated the impact e-learning had on second-year Japanese university EFL students with a focus on improving their TOEIC scores. The e-learning program was provided by Gyuto-e and involved having students answer 800 multiple-choice listening exercises. Four research questions were devised to focus on the score differences after the students had completed the course, between the students who had completed the program and those who had not, the possible gender score differences, and students who had completed the e-learning but had not had any significant TOEIC score improvements. The outcomes indicated that all examinations were statistically non-significant at a 5% level.
Due to the limitations of only one English language class a week, there is little opportunity for the KIT students to dramatically improve their English abilities; however, utilizing e-learning allows for some significant progress. The emphasis now is on furthering the usability of these systems and considering the true pedagogical impacts.