A Scoring Approach for the Assessment of Study Skills and Learning Styles

This paper presents the application of a scoring method and algorithm, adapted from the domain of financial risk management, for the computer-based assessment of study skills and learning styles of university students. The goal is to provide a single score that summarizes the overall intensity of a student’s study skills and, in effect, develop a deeper understanding of the relation between learning styles and study skills. The dimensionality reduction obtained through the scoring algorithm also enables comparing the single-dimensional study skill scores of students for various learning styles. The algorithm computes a weight for each study skill to measure its linear contribution to the overall study skill score, also providing a natural ranking of various study skills with respect to impact on total score. Statistical tests have been conducted to measure the differences in scores for various styles in Kolb’s four-region and nine-region models. The results suggest that students with different learning styles can have statistically significant differences in their overall study skill scores. The primary contribution of the study is illustrating how a scoring approach, based on unsupervised machine learning, can enable a deep understanding of learning styles and development of educational strategies.


I. INTRODUCTION
Education researchers have been interested in identifying the factors that affect college retention and dropout, including factors such as student motivation, self-concept, learning styles, and study skills [1]- [3].
Study skills describe the student's ability in acquiring, recording, and using information and ideas [4]. Study skills include many tasks and psychological variables such as time management, students' information processing skills, setting appropriate goals, selecting appropriate study environment, applying suitable note-taking strategies, concentrating, selecting principal ideas, self-testing, organization, and managing anxiety [5], [6]. Study skills have been shown in the literature to contribute to the prediction of students' academic performances [6]- [8].
Learning styles refer to individual approaches or preferences used while receiving, processing, organizing and using information. They are drawn from the speculations based on behavior observation and personal statements through a variety of learning style inventories (for example, [9]- [11]. [12] indicate that 71 learning style models are identified in the previous studies and the most influential Manuscript received April 15, 2020; revised July 7, 2020. A. Göğüş is with Faculty of Education, Istanbul Okan University, Istanbul, Turkey (e-mail: aytac.gogus@okan.edu.tr). G.
Awareness of study skills and learning styles and the use of suitable strategies can affect student success. Study skills and learning styles are influenced by several social-cognitive factors and are correlated with both intrinsic and extrinsic motivation orientations [8]. Considerable empirical evidence shows that lack of study skills influence academic performance negatively and influence the dropouts from higher education (for example, [5], [6]). From an instructional design perspective, educators realize that knowing the students' learning characteristics gives them many options for engaging in the learning process. Therefore, universities have attempted to design instructions to accommodate students' preferences in acquiring knowledge, enhancing learning, facilitating maturation, and maximizing success [13]. [14] emphasizes that knowing students is an important goal for academic institutions: "Fundamental to the ability to transform the academy is the wisdom and humility to know students, their motivations, their goals, and their learning styles. Institutions cannot merely assume they know their students through the collective experience of faculty and administrators. They must create a culture of evidence." ([14], p.13).
Furthermore, by adapting the perspective of Student Approaches to Learning (SAL) [15], it is possible to bring about more desirable approaches to studying in university students, by using appropriate course design, appropriate teaching methods, and appropriate forms of assessment [16]- [19]. Therefore, some universities offer courses on developing effective study skills for students and offer faculty training on instructional design, development, and evaluation related to students' learning styles. This paper focuses on how learning styles may be associated with study skills, by building upon two earlier lines of research: In the first line of earlier research [20], effects of attributes on the learning styles were investigated through machine learning, for both four-region and nine-region learning styles. For four-region model, the attributes with the highest impact on learning were found to be Planning, GPA, Program (Faculty), Sleep, and Course Schedule, respectively. For nine-region model, the attributes with the highest impact on learning were found to be Activity, Planning, GPA, Study Hours, and Gender, respectively.
In the second line of earlier research [21], which is the predecessor to the current research, a measurement tool based on statistical scoring was developed to investigate the study skills. The most significant study skills which are also presented in this paper were identified.
The current study investigates whether study skills, which are summarized in a single universal score that measures how A Scoring Approach for the Assessment of Study Skills and Learning Styles hard a student works, can be used to expand our understanding of and compare between different learning styles. To this end, the study skill scores are analyzed in association with learning styles, through visual analysis and statistical hypothesis testing.

A. Kolb's Learning Style Inventory
This study adapts the Learning Style Inventory (LSI), which is based on Kolb's theory of experiential learning regarding the various approaches learners choose for perceiving and processing information. The LSI is frequently used as a predictive tool (for example, [22]- [24]). Some researchers have suggested that learning styles are related to culture [25], [26], are discipline specific [27], [28] and differ with respect to various other attributes [29]. In a local context, [30] examine the use of LSI in relation to the Turkish culture.
• Accommodators: concrete, active, hands-on, practical people, leaders, and risk takers. • Divergers: people good at brainstorming, imagining implications, and working in groups. • Northerners: emotional, having deep involvement for reflection, but having difficulty in conceptualizing, making meaning of experience, and organized actions. • Easterners: have a capacity for deep reflection informed by the ability to be both feeling oriented and conceptual, but having difficulty in putting plans into action. • Southerners: mechanical and have highly developed conceptual and analytic capabilities informed both by reflection and action. • Westerners: intuitive and have highly developed action skills informed both by conceptual analysis and intuitive experience without thinking on concrete experience. • Balancing: have adaptively flexible skills, and have balanced learning profiles in both dimensions of the LSI.

B. Scoring
Scoring refers to representing a particular attribute/dimension of an entity (ex: study skills of a student, creditworthiness of a customer) with an ordinal -and preferably nominal-value. Scoring approach is adapted in practically in every domain of sciences and life, due to the simple yet powerful idea it actualizes: Reducing multi-dimensional data about an entity to a single score, that enables comparison with other entities in the group. Examples of scoring include credit scoring in finance [32], ranking of webpages in Internet search results [33], identifying genes from microarrays in bioinformatics [34], and assigning social credit values to citizens [35].
By identifying the need for scientific quantitative approaches for calculating risk scores using survey data, [36] develops a general unsupervised machine learning methodology and an algorithm to determine the weights for the questions of a given survey in the scoring process. The algorithm in [36] is based on successive application of linear regression and, updates weights and scores at every iteration. While [36] applies the developed algorithm in the domain of finance for the scoring of a person's propensity to take financial risk, the algorithm is very generic and can be transferred and applied to other domains. The problem of appropriately combining the values for different questions in a survey into an overall metric is also encountered in education sciences. To this end, the methodology and scoring algorithm of [36] is applied for the scoring of study skills of university students, as in [20].

III. METHODOLOGY
This paper investigates the following fundamental research question: "Is there a significant relationship between students' learning styles and overall study skills?" For answering this question, the mentioned scoring methodology and algorithm of [36], which was initially applied in the domain of risk management, is adapted, similar to that of [20]: Firstly, a single-dimensional score value is computed to summarize the overall intensity of a student's study skills. Secondly, the algorithm results are interpreted to understand which study skills are essential in determining the overall study skill score. Thirdly and finally, the study skill scores for various learning styles are visually represented and are compared through formal statistical tests, to answer the primary research question of whether there is a significant relationship between learning styles and overall study skills. While the process is described in this section, the detailed process is provided in Fig. 1 as a flowchart.

A. Participants and Data Collection
The survey was conducted at a metroplolian international university in Turkey. The university has 3500 students, and the survey was conducted with 418 undergraduate student, 43% being women (n=181) and 57% being (n=237) men. The students' area of the study include two main domains, engineering (n=260) and social studies (n=158). The survey was conducted through e-mail, consisting of three sections: 14 questions on background information, 12 questions on learning style inventory [37], [10], and 50 questions on learning and study skills. The application of each survey took around 30 minutes. The 12 questions were the Learning Styles Inventory (LSI-2) [10]. The version of the LSI-2 used was the adaptation of [37] into the Turkish language, which was shown in earlier literature to be a reliable and valid instrument, with a Cronbach alpha in the range 0.73≤ α≤ 0.88 [37].
The 50 questions on learning and study skills, namely, perception attributes, on a scale ranging from 1 (never) to 5 (always), express how frequently the student applies each study skill. Considering reliability, the internal consistency of the total scale was α≤ 0.83. Deletion of individual items do not result in a drop in alpha below α≤ 0.81, indicating that all items contribute relatively equally to the consistency of the scale. In order to examine possible factor structure of the International Journal of Information and Education Technology, Vol. 10, No. 10, October 2020 scale, a principal component analysis with varimax rotation and eigenvalue of 1.00 as the criterion, is conducted with no limitation on numbers. According to the results of the pilot study, it is concluded that the scale is a psychometrically sound and a reliable measure, as the Cronbach alpha was found to be α≤ 0.83.

B. Kolb's Learning Styles
In the study, The Learning Styles Inventory (LSI-2) [10] is used to examine students' learning style based on the following four sequential elements and their corresponding scores: • A: concrete experience • B: reflective observation, • C: abstract conceptualization and • D: active experimentation The four quadrants of the learning cycle reflect Kolb's four basic learning style types of Accommodator, Diverger, Assimilator, and Converger are created by dividing the C-A and D-B scores at the fiftieth percentile of the total norm group and plotting them on the Four-Region Learning Style Type Grid [30]. For the current study, the cut-off point for the C-A scale is 0, and the cut-off point for the D-B scale is -1. The Accommodator type is defined by a C-A raw score ≤ 0 and a D-B score >1. The Diverger type is defined by C-A≤ 0 and D-B≤-1. The Converger type is defined by C-A>0 and D-B>-1, and the Assimilator type is defined by C-A>0 and D-B≤-1.
Kolb's four learning styles are also expanded to the nine-regions of the Kolb's learning styles, namely Accommodator, Diverger, Assimilator, Converger, Northerner, Easterner, Southerner, Westerner, and Balancing. These regions are defined by dividing the C-A and D-B scores at the thirtieth percentile and sixtieth percentile of the total norm group and then plotting them on the Nine-Region Learning Style Type Grid [31].

C. Scoring
The scoring algorithm takes as input a table that contains the survey data. The I respondents and J questions each respondent answers to form the table with I rows and J columns. The outputs of the scoring algorithm are the following: 1) whether an attribute (a question) should be included in the linear score function 2) weight of each attribute to be included in the score function 3) score of each respondent The initialization step in the algorithm transforms multiple-choice data into numerical values between 0 and 3. In the collected survey data the numerical values corresponding to choices (a,b,c,d,e) are (0.00, 0.75, 1.50, 2.25, 3.00). One important condition here is that for all the questions, the choices should be ordered in the same way. In our study, the value of 1 is defined as corresponding to the least level of a skill, and the value of 5 corresponding to the highest level.
After the initialization phase, the attribute values are fed into the regression based unsupervised machine learning algorithm of [36]. The algorithm operates iteratively, until scores converge. The stopping criterion is satisfied when the average absolute change in scores in the last few iterations is less than the threshold provided by the analyst. At each iteration of the algorithm, a linear regression model is constructed for each attribute, and the response in the incumbent score vector. Based on the regression, weights for the attributes are updated at the beginning of each iteration.
One characteristic of the algorithm is that it allows change in the direction of signs when the choices for an attribute should take decreasing -rather than increasing-values from the choice of 1 to the final choice of 5. Therefore, the algorithm not only eliminates irrelevant attributes, but also suggests the direction of sign for the weights of the attributes.

D. Statistical Tests
A fundamental issue in statistical hypothesis testing is the selection of appropriate statistical tests. For measuring the statistical significance of hypothesized differences in study skills scores, guidelines are provided in [38]. The first decision to be made is whether a parametric (t-test, ANOVA) or nonparametric test (Mann-Whitney, Kruskal-Wallis) should be applied. t-test and Mann-Whitney are used for the comparison of samples in pairs, whereas ANOVA and Kruskal-Wallis are used for the comparison of several samples at once. When applicable, parametric tests are preferred because of their power, their requirement for smaller sample sizes to draw conclusions with the same degree of confidence. However, t-test and ANOVA apply only when each of the samples to be compared follows normal distribution. Nonparametric test such as Mann-Whitney and Kruskal-Wallis, on the other hand, use the rank data to compute the test statistics, and do not require the data to come from a particular distribution [38].
The decision of whether parametric or non-parametric tests should be applied for the comparison of means is made by testing each sample for normality using Shapiro-Wilk normality test [39]. When the hypothesis of normality is rejected for any of the samples, the non-parametric tests are to be used for comparing samples.

A. Scoring Results
The first two results of the scoring algorithm are firstly the subset of significant attributes with respect to predicting the scores, and secondly the weights of attributes in the regression model. The study skills with the highest weights are (S27, S24, S26, S03, S01), referring to the following study skills: • (S27) answering questions of the instructor during the class, • (S24) seeking help from the instructor outside the lecture hours, • (S26) asking questions during class and actively participating in class, • (S03) learning by asking questions during class, and • (S01) learning by listening during the class (Table I).
This is a fundamental insight into what counts in the overall study skills. Six of the 50 questions are assigned a weight of 0 by the algorithm. In the meanwhile, the algorithm removes these six questions from the risk score computations, because they fail to influence the overall scores in a statistically significant way, given the presence of the other 44 attributes, observed in the range (0.29, 1.65). The first 20 attributes in the weight range (1.08, 1.65) can be selected to observe the most preferred study skills and study habits (see Table I).  Fig. 2 and Fig. 3 show the distribution of study skill scores according to Kolb's framework: Fig. 2 displays D-B values agains C-A values. Each point represents a student participant in the study. Darker colors for the points indicate higher study skill scores. One can observe that certain regions have a higher frequency of darker points. Fig. 3 displays study skill scores (deoted by the colors of the points) with respect to the four-region and nine-region styles of Kolb [31].
In this figure, style StyleLSI refers to Kolb's four-region learning styles and style StyleLSR refers to Kolb's nine-region learning styles. In Fig. 3, jittering is applied to prevent the overlap of the points, which have the same style value combinations. The darker points concentrating in some regions can be seen more clearly, calling for formal statistical hypothesis testing.
International Journal of Information and Education Technology, Vol. 10, No. 10, October 2020   Tables II and III show the results of statistical tests for exploring the differences in mean scores for the various learning styles in Kolb's models. In both four-region and nine-region style classifications, only the learning styles with at least 30 observations are included in the analysis. For example, in the four-region model, Accomodator and Divergers are not included in the analysis, because there are less than 30 observations for these two learning styles.

B. Results of the Statistical Tests
In order to select the most appropriate test for the comparison of means of the different learning styles, initially, normality tests are carried out. Table II shows the results of the Shapiro-Wilk normality tests. With respect to the comparison of mean scores (for study skills), firstly, Kolb's four-region model is considered. Because the hypotheses regarding the normality of Converger and Assimilator yield p>0.10, these could not be rejected, suggesting the use of the parametric t-test for the comparison of scores for Converger and Assimilator.
Secondly, Kolb's nine-region model is considered. Because the hypothesis about the normality of Westerner yields p=0.0713<0.10, the hypothesis of this sample being normal can be rejected at a confidence level of 0.10. This implies that, for comparing the means of the scores for each of the styles in Kolb's nine-region model, the nonparametric Mann-Whitney test is to be applied. Table III shows the results of sample comparison tests, where only the statistically significant test results are presented. The results suggest the following two hypotheses: • Hypothesis 1. Convergers have higher study skill scores (better study skills) than Assimilators (supported at a confidence level of 97%, where the confidence level is calculated as 100×(1-0.03)=97%.
• Hypothesis 2. Both Southerners and Westerners have higher study skill scores (better study skills) than Balancing students (supported at a confidence level of 90%, where the confidence level is calculated as 100×(1-0.10)=90%). The two statistically supported hypotheses above suggest that students with various learning styles can indeed differ with respect to their study skill scores. This type of result would not be possible if a scoring approach were not applied. The scoring approach reduces the dimensionality of the problem to a single dimension, enabling statistical comparisons to be carried out and insights to be obtained.

V. DISCUSSIONS
This paper presents the application of a scoring methodology and algorithm for the representation of study skills and learning styles of university students. The goal is to provide a single score value that summarizes the overall intensity of a student's study skills and focuses on how learning styles may influence study skills. The proposed method computes an overall score that represents the study skills, using a linear weighted summary scheme and suggests that instead of using 50 questions, the researchers can use considerably fewer number of questions in future studies. The method computes the study skill scores conveniently and the results can be used to identify students with varying degrees of study skills, and develop strategies with varying degrees of study skills that aim at improving the quality of education.
When learning styles and study skills are compared, the result of Hypothesis 1 shows that Convergers have higher study skill scores than Assimilators. The most preferably International Journal of Information and Education Technology, Vol. 10, No. 10, October 2020 study skills include active learning experiments with hands on activities and interaction with instructors and peers. According to [31], Convergers prefer to experiment with new ideas, simulations, laboratory assignments, and practical applications although the Assimilators prefer reading, lectures, explaining analytic models and abstract concepts. The result of the Hypothesis 1 is in the line with the learning style characteristics discussed in [31].
When the nine-region learning styles and study skills are compared, the result of the Hypothesis 2 shows that both Southerners and Westerners have higher study skill scores than Balancing students. Although most students are at the Balancing style, their study habits and existing study skills fit into the properties of Southerners or Westerners styles because Balancing students have adaptively flexible skills. According to [40], the Southerners reflect on the mechanics of their actions and have highly developed conceptual and analytic capabilities, although the Westerners have highly developed action skills informed both by conceptual analysis and intuitive experience. This comparative result indicates that the study skills cover the learning strengths of these various learning styles.

VI. CONCLUSIONS
The most important contribution of the presented research is proving how a scoring approach can contribute to the higher education sector, by improving the understanding of student study skills and learning styles. At a secondary level of scope and depth, the contributions of the study are: (1) using a scoring algorithm method and technologies in an education research; (2) ranking the importance of study skills in how much they contribute to the overall study skills, and thus improving the understanding of the importance of study skills in the overall picture; (3) showing that students with various learning styles can also be different with respect to study skills scores, with statistical significance.
The preferred learning and study skills in this research indicate that students want to be active learners and want instructors to apply active learning techniques such as collaborating with other student(s), debate, and cooperative/collaborative activities. These findings are consistent with previous studies, which have found that providing opportunities for students to cooperate in peer learning activities during lectures supports the development of their understanding [41] and that students are motivated when activities are undertaken collaboratively [41].
Active learning that allows learners to participate in learning and teaching activities, to take the responsibility for their own learning, and to establish connections between ideas by analyzing, synthesizing, and evaluating, to connect information and concepts to real-world scenarios [42]. In the classroom active learning can be initiated and facilitated through particular instructional techniques, such as exercises for individuals, writing reflections, reviewing others' work, assessing the materials, questions and answers, using the Socratic method, giving immediate feedback, discussions, cooperative groups, developing concept maps, developing comprehensive lists of the concepts, role-playing, group presentations, and games [42]. In addition, peer learning activities during lectures support the development of students' understanding [41]. Active learning has also been linked to higher student motivation and increased confidence with class materials [41].
Furthermore, in earlier literature, active learning has been found to improve student interest in course material, facilitate comprehension and knowledge retention, and allow students to acquire higher levels of learning skills such as synthesizing, comparing/contrasting, and being more independent and a lifelong learner (for example, seeking additional resources, showing less reliance on the instructor, showing more skepticism about research findings reported through the news) [43]. The benefits of active learning in lectures are maximized when tasks are real or reflect how knowledge is used in real life, and when students have opportunities to adopt several roles and reflect on the ideas of others [41], [44]. [45] report barriers of using active learning techniques in instruction such as faculty self-perception, a possible increase in preparation time, the potential difficulty of using active learning in large classes, and a lack of needed materials or equipment and faculty self-concerns, use higher-order thinking, and learn enough content, that faculty members will feel a loss of control, lack necessary skills, or be criticized for teaching in unorthodox ways. Many students prefer opportunities for interaction with instructors and peers, cooperative learning and active engagement in lectures.

VII. FUTURE WORK
The scoring algorithm employed in the current research is an unsupervised machine learning algorithm. Scoring can be through supervised or unsupervised machine learning. In supervised scoring, the algorithm is provided with readily-available scores for observations in a training dataset and is expected to predict the scores for the observations in a test dataset. In unsupervised scoring, the algorithm develops the scores for observations, without any actual scores being given. [46], [47], and [48] illustrate how various other machine learning algorithms can be applied in analyzing data from various domains. One extension to the current research can be applying other machine learning algorithms for a deeper understanding of the collected data.
The field of education research presents a multitude of theories, which can be analyzed further through scoring and other machine learning approaches. To this end, another extension to the current research can be analyzing other theories and models of the field of education research through a scoring approach, and in conjunction with other statistical and machine learning approaches.
Yet another extension to the paper can be using other methods for reducing multi-dimensional data to single-dimensional score. Once such method is Data Envelopment Analysis (DEA) [49], [50], which can be adopted for scoring study skills and other attributes of learners.

CONFLICT OF INTEREST
The authors declare no conflict of interest.

AUTHOR CONTRIBUTIONS
Dr. Aytaç Göğüş collected the data, interpreted the analysis results, and wrote the sections of the paper relating to the theory and practice of educational sciences, as well as International Journal of Information and Education Technology, Vol. 10, No. 10, October 2020 analysis results; Dr. Gürdal Ertek developed the methodology for analyzing the data, developed and coded the algorithm, conducted the data analysis, and created the figures and tables; both authors had approved the final version.