Creating Electronic Portfolio Rubrics for General Education Course Assessments in Thailand

While researchers have introduced and integrated electronic portfolios (e-portfolios) into general education classrooms at Srinakharinwirot University, Thailand, some aspects of workloads and assessments remain unclear to both instructors and students. When evaluating academic performance using e-portfolios, instructors must grade students based on learning outcomes according to the learning objectives and students’ goals. The use of such assessments requires a shift toward realistic and consistent evidence, practice, and reflection. Successfully assessing outcomes and learning processes entails instructors and learners being able to identify and understand all evaluated aspects. Thus far, instructors and learners have practiced and agreed upon two types of e-portfolio rubrics for general education course assessments: an analytic five core skills learning process rubric and an annotated e-portfolio assessment rubric. This study employs indices of item-objective congruence (IOC), item discrimination power, Cronbach’s alpha reliability coefficients, and correlations as analytical tools to determine the efficiency, confidence, and relationship of each item in an annotated e-portfolio assessment rubric.


I. INTRODUCTION
In recent years, the learning outcomes of general education courses in Thailand have undergone significant changes. For example, learning objectives now emphasize cultivating both 21 st century skills and basic ethical principles in preparing the next generation of Thai citizens. In turn, instructors must adjust their pedagogical and assessment methods to ensure that lessons and practices are meaningful and enable students to acknowledge their own improvements. In this study, we examined the application and integration of e-portfolios in the general education courses of Srinakharinwirot University, Thailand. Based on our own past research and other educational studies conducted in several countries [1], it seems that e-portfolios support in-depth, meaningful, and lifelong learning and enhance students' self-directed learning and self-assessment. Thus, they shape students' learning journeys and allow them to understand their own strengths and weaknesses.
Despite e-portfolios' significant value, their successful application currently imposes a heavy workload on both Manuscript received October 9, 2021; revised November 15, 2021. Mebusaya P. Rattiya and Nakazawa Minoru are with the Kanazawa Institute of Technology, Japan (e-mail: rpm.jane@gmail.com, Nakazawa@infor.kanazawa-it.ac.jp).
Panichying Theppharat and Bunjongparu Nanthon are with Srinakharinwirot University, Thailand (e-mail: Theppharat@g.swu.ac.th, nanthon.bun@gmail.com). students and instructors [2], particularly in general education courses. Instructors must devote time to helping students, assessing their work, encouraging them to improve their skills and their understanding of ethics, and consulting with them regarding practice and assessment. Additionally, because general education courses, particularly those in Thailand, typically enroll over 100 students per course, it is difficult for instructors to help and consult personally with each student and continually assess individual students' development. Therefore, it is essential to identify standards and tools to improve the assessment process and create new forms of assessment that correspond to real-life situations involving formative and evidence-based learning. Such formative assessments are more difficult to conduct than standard tests and exams [1]. However, they are essential in enabling learners to understand not only what criteria are being measured, but also how they are being measured.
Our review of e-portfolio use in various settings finds that rubrics are one type of assessment with a strong potential for reducing instructors' workloads. At the same time, rubrics can also help students learn more about themselves and about their instructors' expectations of them. Rubrics can be used to assess students' learning efficiency not only in terms of the learning process but also in terms of the learning outcomes. Finally, they can be applied to diverse types of assignments. Rubrics have been employed to assess both traditional paper-based and electronic assignments.
According to Cambridge [1], the Association of American Colleges and Universities (AAC&U) has established a project called -VALUE Rubric,‖ which aims to create standard rubrics for all higher education e-portfolio assessments. Thus far, the project's rubrics have been employed to assess both hard and soft 21 st century skills, including critical thinking, communication, information literacy, and personal and social responsibility. Although the purpose of the VALUE Rubric project is to standardize assessments within American higher education, it has indirectly created a foundational understanding of sample learning outcomes for stakeholders in higher education worldwide. This benefits students and instructors by reducing the time required to conduct assessments and allowing students to receive clear feedback.
Additionally, Australia and several European countries have researched and established rubric standards for assessing e-portfolios based on national policies and learning outcomes. These standardized rubrics have been utilized not only to evaluate students holistically-i.e., to assess an entire e-portfolio in terms of learning outcomes-but also to evaluate students' efforts throughout the learning process. Researchers confirm that rubrics are a simple and effective way to enhance formative assessment systems for e-portfolio-based learning.

A. Aims and Methods
The purposes of this research were to examine when to assess students' learning and how to create appropriate rubrics to assess students' learning progress and outcomes successfully through e-portfolios, and to measure the efficiency of integrating rubrics into e-portfolio teaching and learning in general education classrooms. We examined previous research on rubrics as well as the stages of the selftransformation learning cycle model in teaching and learning with e-portfolios from our own research. In selecting the types of rubrics, to be used for the assessment formative e-portfolio and for creating the rubrics, the research recruited three general education classrooms instructors, first-year students enrolled in SWU151 or General Education for Human Development, and students' e-portfolios.

B. Data Analysis
After selecting and creating rubrics, focusing on an annotation rubric, we used indices of item-objective congruence (IOC), item discrimination power, Cronbach's alpha reliability coefficients, and correlations as analytical tools to determine the efficiency, confidence, and relationship of each item in an annotated e-portfolio assessment rubric.

III. THEORIES
Before selecting and creating a rubric assessment, we needed to understand and determine the purposes of the assessment and when and how to use it. We took a close look at our past research on the self-transformation learning cycle model, formative assessments, and how to create rubrics to assess e-portfolios.

A. Self-transformation Learning Cycle Model
The self-transformation learning cycle model is the combination of Kolb and Plovnick's experiential learning cycle [3], the PDCA (Plan, Do, Check, Act) of Andrew Walte and Edward Deming [4], and McCarthy's 4MAT learning model [5]. This model has been adjusted to suit Thai students and integrate e-portfolios into general education classes. It contains eight stages: Goal Setting: 1) Inspiring and finding oneself, 2) Identifying strengths and weaknesses; Plan Development: 3) Research and analysis, 4) Plan; Action, Self-Reflection, and Sharing: 5) Action vs. actual plan, 6) Reviewing; Conclusion: 7) Refining, and 8) Final finding.
These eight stages connect to each other with the different objectives, activities, lessons, and outcomes to enable students to learn about themselves through the e-portfolio process. Even though this model works well, it would be more favorable when used for a small group of students. The workload is a drawback to instructors with such large classes as Thai general education classes, which comprise over 100 students per class. Each stage entails activities, practice, and self-reflection, which require guidance and feedback from instructors, especially the action vs. actual plan, reviewing, refining, and final finding stages. Therefore, together with the rubric, students will be able to receive timely feedback and understand what kind of outcome or progress the instructors expect from them.
For instance, the main objectives of SWU151 are to develop certain 1) behaviors, mindset, and intuition as well as 2) cognitive, communication, analytical thinking synthesis, problem solving, with critical thinking skills. This purposely develop basic ethical ideas such as self-discipline, respect, and patience. At the first stage, goal setting, students need to spend some time with themselves, asks themselves questions as part of homework, and interview their peers to identify students' weaknesses, strengths, and motivation to return to the next class. Then, the students choose their goals and write down their motivations. In the plan development stage, students further examine their goals, benefits, and ways to overcome challenges and succeed in the subjects they pick. They need to see and analyze their own learning regarding whether the methods of exercises/practice and activities can be used within and outside the class. Next, they carefully plan their practice with the expected outcome of the plan. Further, during the action, self-reflection, and sharing stage, students start the action, practice, and exercise according to the plan. Sometimes, students are unable to follow the plan or complete it; in this case, they need to determine the reasons and go back to the plan to review it. After exercising and recording the evidence for some time, students need to look back to examine their progress, the selected learning evidence, and their own emotions-how they feel about what has been going on in the past and write down their self-reflection. This is uploaded to their e-portfolios and shared with others-peers and class instructors. These action versus actual and reviewing stages are iterated three to four times every two weeks. This way, students can reflect on their plan and progress as well as exchange feedback. Accordingly, they can encourage each other and share their plans; in case their chosen plans are not suitable for their own learning, their peers can suggest methods or activities to accomplish their goals. In these two stages, students need feedback and assessment to see whether they are on the right path and how instructors have assessed their learning progress. The last two stages are refinement and final findings. Students continue according to the plan that they can refine based on self-analysis and instructors' or peers' guidelines or reinforcement. During the process of modifying the stages, students need to follow a self-checklist as well has had an individual (one-on-one) conference with the instructors. From the comments and self-checking, students can correspondingly refine the plan with reflection. Finally, students need to conclude their lesson through a self-assessment. This includes grading themselves and using all past comments, self-checklist, and evidence to write down details of the self-assessment. All this occurs as part of an exclusive assignment that they can complete at their own pace in parallel with the class lectures, activities, and exercises.

B. Rubrics and e-Portfolio Assessment
Many studies in the past few years have demonstrated that International Journal of Information and Education Technology, Vol. 12, No. 4, April 2022 rubrics can enhance teaching, learning, and assessment in e-portfolio integration [6]. To integrate rubrics into e-portfolio use, we need to carefully examine and select the types of rubrics and create them according to the objectives and outcomes.
There are three types of rubrics. First, an analytic rubric is a scoring rubric that considers each component of the work as a guideline. These guidelines, in turn, are used to grade each part of the work, and each section of the guidelines specifies the assessed items with clear definitions or descriptions. This clarity allows students to understand what is being assessed [7]- [10].
Analytic rubrics have a minimum of one level, but additional levels can be added to accommodate important characteristics or features as the complexity of the assignment increases. An analytic rubric for a writing assessment may, for example, contain only two main items to evaluate: content and language. If the rubric is divided into four levels, however, the sub-characteristics of each level must be specified as well [9], [11]- [13].
Second, holistic rubrics are employed to assess working processes or the work itself as whole. Holistic rubrics do not separate the scoring components. Holistic rubric scoring is simple but can only be used a limited number of times per student. It offers an overall assessment of all operational characteristics. This kind of rating is particularly useful when instructors are interested in diagnosing students' weak points or helping learners improve their knowledge and skills. It enables instructors to better understand their students by observing the entire learning and working process [8], [10], [14].
Lastly, in annotated holistic rubrics, the instructor evaluates the work as a whole before reevaluating specific parts [15]. The results of such assessments are used to inform reflections regarding some student characteristics. Annotated rubrics address the limitations of holistic and analytic rubrics by combining their distinct features. This method begins by evaluating the -big picture‖ of each student's work using a holistic style. Then, the evaluator may choose some characteristics of the work to evaluate in greater detail using an annotated style. These annotated evaluations do not impact the holistic score, nor are used to diagnose defects in any characteristics of the work. Rather, they simply offer students more specific feedback.
In brief, an effective rubric should be composed of [9], [13], [16]: 1) Task description: The instructions of the assignment that instructors want students to complete. With the task description, students will learn and understand what the assignment is, the instructor's expectations for it, and how it will be assessed. 2) Scale or criteria: The criteria determine the main points for evaluation, such as the learning outcomes, the learning objectives, or the standards of each outcome in each learning project. 3) Dimension or performance levels: These levels should use mostly odd rather than even numbers to prevent scoring that falls in the middle, which could hinder efforts to create clear ability or performance divisions within each level. Additionally, each level may be defined in numbers, words, or both to indicate the quality-for example, -Excellent,‖ -Good,‖ -Fair,‖ and -Does not meet criteria.‖ 4) Quality or performance description: The expected quality and its corresponding indicators must be defined in a way that is clear and simple enough for others to understand and distinguish between levels.

IV. PROCESS
Before creating rubrics, researchers and instructors examine the eight steps of self-transformation learning cycles, defining both objectives and outcomes in each stage to select the type of rubric and determine its objectives. The objectives of creating rubrics are as follows: 1) To evaluate students' learning processes and outcomes.
Learning process evaluations include goal setting, plan development, evidence collection, self-reflection, a final summary, self-assessments in each period, and their connection. After the results are evaluated, the final e-portfolios are reviewed and assessed.

2)
To evaluate the e-portfolio process, its organization, the individual identity of students, and its final outcomes. Therefore, we and the instructors decided to create two types of rubrics to use in the last four stages (action vs. actual plan, reviewing, refining, and final finding) as well as at the end of the courses. An analytic rubric would assess students' learning processes for the goal setting, plan, evidence, and self-reflections to see how students progressed in their learning and to keep them working on their own practices. Depending on the skills and ethical principles that the students chose as their goals at the beginning of the classes, six analytic rubrics were produced and used for the last four stages of the model: a communications skill rubric, critical thinking skill rubric, information communication and technology (ICT) rubric, information literacy rubric, personality rubric, and ethical practice rubric. These skills and ethical principles were set and introduced to the students at the beginning of the class. They are basic requirements of the general education national curriculum [17]. An annotated rubric, on the other hand, was created and used to assess the final e-portfolio as a comprehensive learning outcomes assessment.
For the former, we began with a task description to ensure that the students understood what they had been assigned to do and what would be assessed. Then, we determined the criteria for soft skills/ethics and hard skills. Performance levels were separated into four characteristics, and each student's learning improvement was ranked as -No progress,‖ -Progress,‖ -Awareness,‖ or -Thoughtful.‖ These words were carefully chosen to motivate students and remove any negative connotations that might have discouraged them from progressing in their learning. The four levels corresponded to the following rating scores: No progress (5), Progress (10), Awareness (15), and Thoughtful (20). Finally, we provided clear descriptions for each level regarding what would be assessed and how students could reach the level.
In devising the ethical principles (soft skills) assessment, it was difficult to decide upon clear definitions for how students might improve their habits, beliefs, and ethical principles, and this difficulty, of course, complicated the evaluation process. Importantly, however, the instructors always remained cognizant of this difficulty and its potential to discourage student learning, and in designing all learning processes and assessments, the instructors attempted to simplify the assessment process as much as possible.
First, after studying ethical and moral theories, we used Kohlberg's hierarchy [18], [19], which is consistent with ethical principles in Thailand. We and the instructors agreed to utilize three levels from the inside-out perspective. These three levels were 1) human rights-considering others, an environment, or a situation from a self-perspective; 2) human efforts-considering others, an environment, or a situation from a self-perspective and from the perspective of others who belong to the same environment and/or small society; and 3) human experiences-expanding the perspective and behaviors to include social impacts and experiences.
We attempted to break down and provide clear definitions for ethical and soft skill factors stemming from both 21 st century skills and Thai principles, mostly based on religious principles. Most of these ethical and soft skills overlapped, and each factor could have been the result of one of the other factors. After reviewing and analyzing all theories and studies, we, the researchers, and instructors, agreed upon three main contributing factors from which students could select sub-units/ethics. These factors were regulation, patience, and consideration. All the factors and levels were defined, and the learning to be assessed was explained as shown in Table I below. The second type of rubric, created to assess the e-portfolio as a comprehensive learning outcome, was an annotated rubric. Although the instructors employed this rubric to assess the entire e-portfolio, they also used it to assess the following 15 distinct factors and aspects: 1) Goal Setting: Students establish realistic, appropriate, and attainable individual goals. 2) Motivated to Learn and Grow: Students are engaged in their own learning and development of individual competency and propose their own ideas to promote self-flourishing.

3) Record Keeping: Students complete all weekly
individual evidence accordingly, ensuring consistency with their plans and goals. 4) Task Completion: Students individually, thoughtfully, and effectively complete the necessary preparation and planned tasks in a way that is consistent with the required styles (records, reports, evidence, etc.). 5) Self-Reflection and Assessment: Students reflect on their self-transformation and cognitive-affective-behavioral congruence and make a summary describing themselves. 6) Self-Awareness and Self-Understanding: Students establish an awareness and appropriation of individual beliefs, systems, values, needs, limitations, strengths, weaknesses, and ethics to gain self-insight and enhance self-acceptance. 7) Attendance: Students demonstrate strong attention and enthusiasm in activities or tasks based on individual plans (i.e., a variety of evidence, frequent updates, results, and references). 8) Appraisal and Positive Feedback: Students demonstrate the ability to provide appropriate and constructive feedback and suggestions to their peers. 9) Openness to Feedback: Students respond non-defensively and alter their behavior in accordance with instructors' and peers' constructive feedback. 10) Flexibility and Adaptability: Students demonstrate the ability to be flexible, alter their methods, and adjust plans to accommodate unexpected circumstances, events, and situations. 11) Knowledge: Students demonstrate an understanding and appreciation of the core skills that they individually choose to develop. Additionally, students demonstrate their ability to understand and use the empirical evidence and information they derive from research on their chosen topics. 12) Autonomy and Identification: Students recognize their own authenticity and personalities, which they demonstrate through their profiles and e-portfolios. 13) Maturity: Students demonstrate self-regulation, honesty, and transparency while developing their e-portfolios (i.e., references, honest reflection and feedback, self-acceptance, etc.). 14) Multimedia and Technology: Students demonstrate multimedia and technology skills, which they apply to collect evidence in appropriate and varied ways. 15) E-Portfolio and Organization: Students organize their e-portfolios appropriately and creatively. In evaluating students on the above criteria, we used the same four levels as those used in the holistic rubric, namely -Thoughtful,‖ -Awareness,‖ -Progress,‖ and -No progress.‖ These rubrics were introduced to 84 first-year students enrolled in SWU 151 or General Education for Human Development. Through practice and completing the e-portfolio, students understood what they needed to do and what instructors expected from their e-portfolio. They also utilized the analytic e-portfolio rubrics to assess themselves and their peers. This enhanced students' relationships with each other and with their instructors. The rubrics provided clear instructions and stimulated the students to bring themselves and each other to follow the plan to reach their goals. Both the analytic and annotated rubric for e-portfolio assessment built the students' confidence in what they were doing.

V. RESULTS AND DISCUSSION
After creating rubrics for the learning process and e-portfolios, we asked three instructors, who specialized in different fields and taught different classes, to review and measure the rubrics using indices of IOC. The IOC scoring ranged from -1 to 1. Shown in table II, the scores for most items on both rubrics equaled or exceeded 0.5, and these items were, therefore, retained. This means that these items were associated with the objectives of integrating e-portfolios into learning and improving general education outcomes (i.e., the development of students' core skills). Focusing on the annotated rubric for e-portfolios as a comprehensive assessment, we used item discrimination power to evaluate the success or failure of each rubric item.
The item discrimination power, shown in table III, ranged from .30-1.00, indicating that every item was effective and could be used to discriminate between e-portfolio learners. The lowest scoring item was Knowledge, while the highest was Attendance. After using item discrimination power to distinguish between effective and ineffective ability measurements on each topic, the researchers employed the Cronbach's alpha reliability coefficient to determine the confidence and relation of each item as a whole in table IV. At 0.932, the overall Cronbach's alpha reliability coefficient indicated significant confidence and effectiveness.
Next, we proceeded to determine the correlations among each of the 15 individual items. Fig. 1 below presents the detailed results.
As shown in Fig. 1, most of the correlation coefficients approximated or even exceeded 0.5, indicating that the items were significantly related to one another. Below, we present each item and its correlations. 1) Goal Setting was significantly related to and affected Maturity at 0.548, Motivated to Learn and Grow Initiative at 0.541, and Self-Awareness and Self-Understanding at 0.503. 2) Motivated to Learn and Grow Initiative was significantly related to and affected Self-Awareness and Self-Understanding at 0.543, Maturity at 0.493, and Task  and affected E-Portfolio and Organization at 0.401. Based on these scores, all items were related to each other and enhanced learners' abilities in other areas. Improvement in maturity, in particular, was strongly related to improvements in most other items. This highlights how integrating e-portfolios into teaching and learning significantly impacts learners' core skills, particularly their understanding of ethics. We further attempted to measure the correlation between the analytical ethics rubric and the annotated e-portfolio assessments and found that e-portfolio processes (especially Motivated to Learn and Grow Initiative, Multimedia and Technology, and E-Portfolio and Organization) significantly enhanced students' ethical learning.
Conversely, Knowledge was not significantly related to any of the other 13 fields, and its independence is consistent with the item discrimination power described earlier, where the Knowledge field had the lowest range. The independence that Knowledge exhibited was also evident with other less significant fields, such as Autonomy and Identification, and is likely due to the students in the sample being relatively unfamiliar with e-portfolios. In fact, the sample of this study included first-year students who had never used e-portfolios and who had yet to be introduced to teaching and learning in general education or other subjects. Prior to our study, portfolios had been used only to outline the purposes of students' products in the fine art faculty but not in other faculties for learning-based teaching, learning, and assessments. Therefore, if the students and other instructors continue practicing and integrating e-portfolios in other courses, these two fields of the annotated e-portfolio assessment rubric will likely exhibit drastic progress and strongly impact students' learning. Indeed, students must often be exposed to and practice hard and soft skills for an extended period of time before fully comprehending them.
According to Thorndike's [20] learning theory, if one were to measure the learning process and draw a graph of erroneous and correct responses, the steepest part would appear on the left. Thus, trial and error is essential for students to learn through experience [21]. The more students learn, practice, apply, and experience through their lessons and their daily lives, the more their learning will progress and the more this progress will be evident in their e-portfolios. This progress (and students' ability to recognize it) in turn shapes their identities, learning journeys, and beliefs. In fact, real-life cases support the assertion that e-portfolios clearly demonstrate learners' knowledge, skills, personalities, beliefs, experiences, and identities [1]. However, further confirming this assertion and adequately measuring the Knowledge field, in particular, requires that students continue using their e-portfolios in other courses.

VI. LIMITATION AND CONCLUSION
The developed rubrics were demonstrated to help both students and instructors effectively integrate e-portfolios into the teaching, learning, and assessment processes of general education courses. Thus, these rubrics enhance the process of teaching and learning and reduce instructors' workloads. Although most of our findings demonstrate these rubrics' positive effects on students' learning processes, current efforts to integrate e-portfolios into general education classes in Thailand have room for improvement in many areas.
The issues requiring further study and research to enhance the progress of integrating e-portfolios into undergraduate general education classes in Thailand can be divided into three areas: 1) The effectiveness of integrating e-portfolios into the process of teaching and learning ethics and soft skills, as well as the value of using rubrics in their assessment Some other fields in the rubric, particularly Knowledge, require additional research to confirm students' learning progress. Examining both past and current research reveals the ways in which students' attitudes toward and understandings of their learning in general education have changed through the process of practicing and completing e-portfolios. Rubrics make it easier for students to understand and draw connections for themselves between knowledge, skills, beliefs, motivations, and goals. This, in turn, helps them create value and meaning in their own lives now and in the future. Ethics and soft skills are among the most difficult subjects to assess. Hence, it would be useful to study more deeply the concrete results regarding e-portfolios' effectiveness in ethics and soft skills subjects. 2) Methods for integrating technology into rubric assessments and e-portfolios As mentioned in the introduction, many researchers have found that integrating e-portfolios into curricula and classrooms is effective and meaningful for both students and instructors. However, it can also increase instructors' workloads. Although we created rubrics to reduce assessment time, the task remains quite time-consuming for education instructors, especially in Thai universities where classes often enroll more than 100 students, and each instructor must teach at least three to four classes per term. Under such circumstances, instructors simply cannot individually assess the progress of each student in class content, learning outcomes, skills, e-portfolios, etc. Therefore, researchers must work to automate checkpoints via e-portfolios to alert and advise students on their learning progress. Such efforts will improve students' awareness of their own progress and reduce instructors' workloads.
Moreover, data from students' e-portfolios can be useful for students (in terms of understanding themselves and creating their future paths), instructors (in terms of improving their teaching and learning), school administrators, and even future employers. 3) Methods for enhancing e-portfolio teaching and learning management To integrate and assess their own learning progress via e-portfolios, students require technical and learning support from instructors. Of course, offering this support further increases instructors' workloads. Therefore, additional support from school boards is necessary. In the case of Thai universities, we believe that adopting and integrating e-portfolios into the general education program will expand opportunities for successful learning. Moreover, to enhance students' learning through e-portfolios, we recommend that universities establish help centers that both students and instructors can consult. The establishment of such centers also has the potential to reduce instructors' workloads.
Finally, rubrics are highly effective for use with e-portfolios to assess learning progress and learning outcomes. The combined use of rubrics and e-portfolios enhances students' understanding, expectations, and self-assessments while enabling instructors to clarify their own expectations and evaluate students' progress with the same standards. This can also reduce instructors' workloads. The rubrics we developed indicate students' learning progress and the relationships between their learning progress in various fields. They also helped the students recognize their own areas for improvement. However, many issues require additional research before the successful integration of e-portfolios into teaching and learning.