Validation of ILS through Exploratory Factor Analysis and Internal Consistency Reliability for Secondary Grade

There is a continuous debate on the validity of learning styles in literature. Several models have been proposed for the learning styles, corresponding to the psychometric assessment instrument. Among these models, Felder-Silverman model is widely used by educators to identify the learning styles of the engineering students. The instrument that measures the learning styles is the index of learning style (ILS). This study focuses to identify the validity and reliability of the ILS instrument for middle school students (N=260). This includes internal consistency reliability and construct validity report of the ILS. As a result of the study, the reliability of the instrument was established, however, it was found that there were cross loadings in the 14-factor solution and the 4-factor solution. Thus, the instrument validity for secondary grade students was not established.


I. INTRODUCTION
Over the past 40 years, there is an emergence of various learning style models which claim that students learn better according to their preferred styles. The better a teacher understands learners" differences, the more s/he can accommodate their learning needs [1]. Among others, the Felder-Silverman model, initially presented in 1988, has been widely used by the researchers in wide range of disciplines including medical, engineering, linguistics and technological education. This model is based on the learning styles of the engineering students and proposed that each learner has specific learning preferences over multiple bipolar axes. The bipolar dimensions include, Active/ Reflective (AR), Sensing/ Intuitive (SI), Visual/ Verbal (VV), Sequential/ Global (SG), and Deductive/ Inductive (DI).
The Felder"s experiential, phenomenological and sensory model is not completely original since its dimensions, such as, sensing or intuitive are taken from Jung"s theory, active or reflective dimension is taken from Kolb"s experiential model, and the visual/verbal dimension is derived from Dunn and Dunn model of learning styles [2]. The Deductive/Inductive (DI) dimension, however, was not included in this instrument because it supports the traditional teaching method and can provide incentives for continuing use of traditional DI instruction. According to Rahadian and Budiningsih, there is a need to use the Felder-Silverman model to identify the preferences of middle school learners, so that the teaching Manuscript received April 22, 2020; revised January 12, 2021. The authors are with the National University of Sciences and Technology (NUST), Islamabad, Pakistan (corresponding author: Khawar Khurshid; e-mail: khawar.khurshid@seecs.edu.pk). and medium of instruction can be altered according to learners" requirements [3].

A. Felder-Soloman Index of Learning Styles
The Index of Learning Styles (ILS) by Felder-Soloman provides the information of learning preference along four bipolar characteristics AR, VV, SI and SG, which are related to processing, input, perception, and understanding, respectively. This diagnostic tool was developed in 1991 by Richard. M. Felder, and Barbara A. Solomon. The learner can have neutral, moderate or strong preferences for the modalities on the basis of scores. ILS is a free, concise and comprehensive self-administered 44-items tool to identify the learning style in light of the aforementioned dimensions of educational experience (i.e., processing, perception, input, understanding). Each dimension is further allocated in contradicting learning style. The validity of the ILS has been reported by various studies [4]- [9]. Each bipolar characteristic in terms of teaching styles is briefly described as follows: Content: Type of information emphasized by the teacher can be concrete (facts based) or abstract (theoretical).
Presentation: The way a teacher presents the information can be visual (pictorial, graphical etc.) or verbal (lecture, discussions or text).
Student Participation: The way the learners are allowed to participate can be active (talk and discuss) or reflective (observe, think, listen and watch).
Perspective: The perspective taken to present the information can be sequential (step by step) or Global (contextual, divergent thinker and synthesis).

B. Index of Learning Scale Scoring
Felder-Silverman learning style model provides two inclinations for each dimension. There are 11 items for each dimension, which helps in identifying the learner preference in the bipolar spectrum. For example, regarding the input dimension, there are two bipolar preferences provided, visual or verbal, and there are 11 items for the input. The items are developed in a way that the learner has to choose between either "a" or "b".
The "a" choice in these 11 items correspond to "visual", while the "b" choices correspond to "verbal". Each item is scored 0 or 1 based on the selected choice. The sum for each dimension is calculated separately for the "a" and "b" selections. The difference in the cumulative values of "a" and "b" helps to identify the preference in this dimension. For example, if a learner, under "visual/verbal" category, has 5 "a" and 6 "b" selections, then the response would be "1b". Hence, the scoring of ILS is 1, 3, 5, 7, 9 and 11, where 1 and 3 show a balance along the bipolar continuum, 5  moderate inclination whereas 9 and 11 show a high inclination. The measurement of validity and reliability is essential before the use of instrument for research study. These measurements are required beforehand for the interpretation of study findings [10]. Although there are several studies which report the validity and reliability of ILS for higher education [7], but we could not find reliability and validity report on middle school students. Thus, the present study has been conducted purposely to explore the most essential prospective of the instrument for secondary grade students. It is equally important to know the validity and reliability of tool before making any decision on data collection through such tool.

II. RELATED WORK
The Felder-Silverman model is widely used to identify learning styles in higher education engineering students. However, it is also used to determine the learning styles of non-engineering students. Hosford and Siders investigated the internal-consistency reliability, factor structure and temporal stability of the tool for medical students [8]. Wang and Mendori used the model to identify the learning preferences of 138 language students and 60 engineering students [9].
Felder provided several strategy suggestions for teaching students, so that range of learning styles can be addressed through instructional approaches [1]. According to Felder, to provide balanced instructions, learning styles need to be examined. The educators can benefit from using learning style model for curriculum designing, developing educational technological tools, instructions, skill development, such as, communication skills, leadership skills, interpersonal skills etc. [11]. Concept of Felder-Silverman examines the learner styles using 44-itemed tools: index of learning style (ILS). There are four bipolar dimensions and nearly all of the questions in the ILS are representative of their dimensions [12]. The free of cost web-based version of ILS is used extensively to identify the learning styles of students every year [7].
Graf et al. analyzed the Felder-Silverman learning style model by dividing each dimension in semantic groups through hybrid approach [13]. The focus of the study was to analyze the model for higher education students in understanding their learning preferences for technological education and online environment. Litzinger et al. examined the reliability, factor structure of ILS [5]. Moreover, the ILS construct validity was also analyzed through direct student feedback about their responses regarding the learning styles. Five option response scale was developed and tested for reliability and validity and it was identified that changing the response scale does not change the factor structure but positively impact the scale reliability [6].
Zywno recommended further empirical evidence requirement for the validity of ILS after analyzing the psychometric properties of ILS for hypermedia assisted learning environment [4]. He identified the overlap in two dimensions (i.e. sensing/intuitive and sequential/global domain) and established that ILS allows to assess separate quantities based on Felder-Silverman model. Felder and Spurlin examined the research studies that reported the validity and reliability evidence of ILS and suggested that ILS is a valid and reliable instrument to assess student"s learning preferences for balance instructional designing [7]. Litzinger et al. identified eight factors through factor analysis in validity analysis in 2005 ILS validity and reliability study [5]. Although, some items did not load well onto any factors, however, the authors still established that the construct validity revealed factors that matched with theoretical construct of the model [5]. In another study by Hosford and Siders, ILS was used to identify learning styles of medical students and four factors were identified whereas few items were not loaded appropriately with any factor [8].
The Solvak version of ILS was created to report the reliability of instrument for 37 high school psychology students [11]. Kaliska determined the reliability of Sovak-version-ILS in term of stability in time and reported non-significant correlation. However, insignificant numbers and weak motivation of participants, and poor translation of tool may have caused these results. Additionally, validity report of the instrument was not provided by the study. In another study done by Wang and Mendori, Mandarin version of ILS was used to explore the gender-based learning preferences differences [9]. It was established that statistical value of internal consistency for the mandarin version fulfills the acceptability criteria.
According to Flake et al., examining the on-going validation of the instrument is prerequisite, even if the existing scale is used in a new context or population [10]. However, ILS was used to determine learning styles of secondary grade students in a recent study by Rahadian and Budiningsih without determining reliability and validity based on empirical evidence [3]. The present study, therefore, examined the average item correlation and exploratory factor analysis to deliver the empirical evidence of reliability and validity of instrument for secondary grade students.

III. METHODOLOGY
This study explores the internal consistency reliability and the construct validity of Felder-Silverman"s ILS for the secondary grade students.
Internal Consistency Reliability: Internal consistency is an important measurement for the instrument to examine a uniformity of construct. Reliability information is mostly reported using the Cronbach"s alpha coefficient value, which shows the covariance level among the questions or items of the instrument [10]. If the scale has high correlation between the items, the alpha (α) value also increases. Reliability is a necessary component for the instrument to be standardized, but it is not sufficient for the validity of that instrument.
Construct Validity: Validity means measuring the degree to which the instrument provides consistent results. According to Cronback and Meehl "construct validity is involved whenever a test is to be interpreted as a measure of some attribute or quality which is not operationally defined" [14].

A. Participants
The study for the validity and reliability of ILS was done on the data collected from the secondary school between the age of 11 and 16 years, with (N=260) participants for the study. The grades of the students were between 6 and 10. The English language form of ILS was used having 44 questions. Signed consent forms were obtained from the parents prior to the study, and the verbal consent was taken from the students. Thus, participation from all the students in this study was voluntary. It was made clear to the students that if they do not wish to participate, they were not exposed to any risks. Some students (N=18) requested to opt out of filling the ILS survey forms in favor of using the duration as free time.

B. Research Design
To validate the Index of Learning Style (ILS) instrument, for secondary grade students, two analyses were conducted. Average items correlation was identified by Cronbach"s alpha coefficient to find the internal consistency reliability of an instrument. Construct validity of the tool was identified using exploratory factor analysis. Researcher should provide the evidence of construct validity, even the reliability of the instrument has been established [10]. Exploratory Factor Analysis (EFA) provides the necessary count of factors that are representative of underlying construct. It is a measure to examine the dimension of a group of items.
The instrument was administered at the start of the session of an academic year 2018. This allowed student to respond to the instrument in more relaxed manner since the workload for the students was negligible. The paper pencil form of instruments was used for the study. This made it easy for the students and teachers (who helped in conducting the survey) as they were able to obtain the data in their classes and did not require going to the computer lab. For construct validity and internal-consistency reliability, 240 valid ILS surveys were utilized. Initially, the scale scoring was done manually on the form, and later the data was digitized to be used in Statistical Package for Social Sciences (SPSS) version 20.

A. Internal Consistency Reliability
The internal consistency reliability analysis of the scale was performed on 240 valid instruments. Cases (N=20) with missing responses were excluded from the study so that the analysis was not conducted on missing data. The internal consistency reliability of index of learning scale is given in the Table I. It was found that the Cronbach"s alpha value ranged between 0.515 and 0.616, which is compared with other studies to identify the validity of the instrument as shown in the Table II. Although the other studies were done on the graduate or post graduate students, the reliability report of ILS is quite similar.
The extent to which all items in the test measure the same concept (internal consistency reliability) of the Index of Learning Style for secondary grade students was identified through Cronbach"s alpha. Since the construct validity must be explicitly analyzed, even if the reliability of the instrument is established, therefore, exploratory factor analysis was performed for the ILS used for secondary grades to extract the factors using principal component analysis (PCA). The purpose was to establish the construct validity of the ILS. The construct validity is suitable for all kinds of psychometric test [14]. Kaiser"s criterion method was used for the factor analysis, and the associated Scree plot is shown in Figure 1. This helps in deciding the number of factors that can be retained using the eigenvalue of a factor, representing the amount of total variance of the factors [17]. Fourteen factors solution was estimated as the Kaiser"s criterion method sometimes extracts too many factors. The results of the 14-factors analysis are shown in Table III. Also, four factors were examined with 35.68% total variance, and the associated results are given in Table IV.  In fourteen-factor solution, three of the four scales supported the relative orthogonally, where Sequential/Global scale pre-dominantly loaded on Factor 4, items from Active/Reflective scale loaded on Factor 1, and Sensing/Intuitive loaded on Factor 2. However Visual/Verbal loaded on multiple factors as shown in Table 3. For comparison, the factors were reduced to four using Kaiser Normalization method. The four-factor solution is given in Table IV. The solution greater than 0.4 is considered as high loading, because the correlation coefficient ±0.40 is considered important and ±.50 is considered significant [18].   III: THE HIGH LOADING OF THE 14-FACTOR SOLUTION  Factors  AR  SI  VV  SG  1  6  2  --2  2  5  3  -3  ----4  1  --7  5  2  1  --6  --2  -7 - The results of the extracted factors, using principal component analysis, show the relative orthogonality of one out of four scales of ILS, where the items from sequential/ global are loaded on factor 4, visual/verbal scale is loaded on factor 2 and items from active reflective scale and sensing/intuitive scale are loaded on all four factors, respectively. In four-factor solution, it is observed that items from sequential/global predominantly mapped on factor 4, and these findings have similarity with 14-factor solution shown in Table 3. Most of the items in verbal/visual load on factor 2. Seven items (i.e. 12, 15, 23, 24, 39, 34 and 35) poorly loaded on factors, and had correlation coefficient less than 0.3 therefore not reflected in the table as suggested by Tabachnick and Fidell that in order to inspect the factorability, correlation coefficient over 0.3 should be considered [19]. Loading less than ±0.3 is considered minimal value as a rule of thumb [16]. Nine items out of 44 had cross loadings which are 1,2, 21, 22, 37, 38, 41, 42 and 44, as seen in Table IV. The items with poor values are highlighted using asterisk sign (*), and cross-loadings are highlighted using bold font in Table 4. There was cross loading of items, which was observed in 14-factor solution and also can be seen in the 4-factor solution in Table 4. Practically when there are cross loadings of any item in several factors it should be altered or deleted to avoid cross loading. This will ensure presence of strong construct validity among items.

V. CONCLUSIONS
The index of learning styles (ILS) is widely used to identify the learning preferences. The instrument is designed to identify learning styles of engineering students. However, ILS is also used to determine learning styles of secondary grade students in a few recent studies. In these studies, types of instructions and media are identified based on learners" style for the secondary grade students. The instrument validity and reliability were not established previously for the secondary grade students. This study examined the reliability and validity of the ILS for the secondary school students. The internal consistency of the ILS was found appropriate since the Cronbach"s alpha value of all the items was greater than or equal to 0.5, which is the acceptable threshold to ensure the reliability of the instrument. The results based on internal consistency reliability were in-line with the previous studies [5], [6], [9], [15], [16]. The difference, however, is that all of these studies found the internal consistency reliability of the instrument for higher education students.
The exploratory factor analysis was done to extract the factors and factor loading against each item. It has been identified that few items had cross loading which is aligned with the study done on engineering and business student by Livesay, et.al, and Zwanenberg, Wilkinson and Anderson, [15], [16] and few items do not align well with any factor. However, Litzinger et al. in [5] also found that few items such as 17, 39, 40, and 42 loaded poorly with value less than 0.3, though in present study, items that poorly loaded are different except item 39 which is common in both studies. Another study by Zywno, done on the engineering students found cross-loading between three out of four dimensions i.e. Active/ Reflective, Sensing/Intuitive and Global/Sequential [4]. The revision in tool might impact the scoring system of instrument as suggested in [5]. However, the new system of scoring might cause confusion as the tool has been used in several research studies and for numerous individuals.
It is established that reliability of the instrument is within the range of acceptability and deletion of any item would affect the reliability and will also require the change in the scoring of instrument. To improve the construct validity in the present study, it is suggested that the instrument is administered with larger sample size of secondary grade students to minimize cross loading effect. If large sample size does not address the cross-loading issue, then the instrument will require a revision. Currently, the analysis of the study was performed on 240 valid instruments. In a future study, the instrument will be administered on more students of the secondary grade. Moreover, test/re-test reliability will also be included in the subsequent study.

CONFLICT OF INTEREST
The authors declare no conflict of interest.

AUTHOR CONTRIBUTIONS
Munazza Ahmed Mirza analyzed the data for exploratory factor analysis, wrote and presented the paper. Khawar Khurshid. helped to compute internal constancy reliability required in the research. He also supported in improvement of the structure and format of the paper with respect to the guidelines of the publication. Kinza Sohail helped to approach and attain permission from the research site (i.e. private school), provided her support in recruitment of participants in the study. For this purpose, she ensured to contact all the parents to attain informed consents. Salman Biland provided his support in conducting literature review, data collection and data entry in SPSS required for the study. Asma Ahmed Mirza helped in developing methodology of the present study.