Digital Learning Object for Audiovisual Production

This article deals with a case study in which a digital learning object (DLO) was developed to assist in the pedagogical practice in higher education (audiovisual area). The main results obtained were the excellent conceptual evaluation received by the DLO tool; as well as great concepts received in evaluations that refer to relevance, differentiation, credibility, and intention to use (among other metrics). In addition, a blind analysis also showed that there was no qualitative difference between the practical work developed with or without the aid of the tool (due to a potential gain of time that could be perceived and enjoyed for the execution of the activities performed, due to an automation process offered by the tool). The final conclusions pointed to a positive indication of the use of DLOs in teaching practice in higher education, as the digital tool was very well received by students during classes and helped to review and reinforce the learning content taught. Thus, the study reinforces the research developed in the area of education on the effectiveness of the use of technologies in supporting pedagogical activities, besides adding another experiment related to mixed digital learning. However, it stresses that collaborative research can lead to a further analysis of the pedagogical contributions of DLOs.


I. INTRODUCTION
Innovation can now be a way of improving the quality of education, helping to overcome productivity and efficiency crises, and changing the view that the industry is sometimes perceived as resistant to change. Digital learning modes are becoming increasingly used in higher education to help students learn, encourage collaboration, creativity and provide students with the tools and skills they need to work and live in an increasingly technological and digital world. This practice-based case study highlights elements used to create a digital learning object (DLO) and explores students' perceptions of the role of a DLO in learning.

II. OBJECTIVES
This article reports a case study project with three objectives; develop a DLO to assist in performing practical exercises for the production of visual effects (VFX) in audiovisual (where there is the interaction between real and virtual images -match-moving); evaluate the effective pedagogical contribution that DLO offers through research with students in the classroom (undergraduate level); enable the application can also be used outside the educational context, as a tool to aid audiovisual productions with such technical characteristics.

A. New Technologies in Teaching -DLOs
The new technologies are becoming more and more a part of daily life and the area of education is one of those that is also transforming due to technological advances. Currently, several types of research are carried out to study how technology can and is being included in teaching and which of these forms can bring some kind of pedagogical advance and contribution or are only a distraction that does not contribute to the learning process.
"The introduction of new forms of media to the classroom-whether they consist of charts and diagrams, textbooks, films, or various media accessed through computers-has not only changed the process of teaching and learning, but also directly affected schools themselves as state institutions. The implementation of these media is regulated by the state in order to enforce its educational monopoly within the classroom and to control the nature of knowledge. At the same time, the implementation of these media provokes intense debate among educators, who dispute the extent to which the new media improve the quality of teaching and learning or even harm children." [1].
There are currently several concepts and definitions about learning objects, such as Sosteric & Hesemeler, "A learning object is a file (image, movie, and so on.) that is intended to be used for educational purposes and has, internally or through associations, suggestions on the appropriate context for its use" as cited in [2].
Some authors define learning objects as any resources used to aid learning. Wiley define them as digital resources used for educational purposes. "Any digital resource that can be reused to assist learning" [3]. His definition includes any digital resource that can be distributed over the network, on-demand, be it small or large.
"Educational objects can be defined as any resource, supplementary to the learning process, which can be reused to support learning. The term learning object generally applies to educational materials designed and constructed in small sets to maximize learning situations where the resource can be used." [4].
Likewise, the digital learning term can be understood as the "Use of digital technology to support learning. This term is context-free to specific digital technology, environment, pedagogy, instructional design, and learner interaction with the material or environment." [5].
"New generation classrooms and neither alone digital technology are not possible to imagine in education without educational content. These contents are different forms of digital learning objects (DLO), respectively specific software for their creating and distribution. New Media Consortium (NMC) defines digital learning objects as a group of materials (texts, hypertexts, graphics, pictures, simulations, films, sounds, etc.) which is reasonably structured and is based on educational aims and objectives. It is multimedia content, educational content, educational software or software instruments used in computer-supported education. DLO can be understood as a category of multimedia learning aids. Dostá l defines multimedia learning aid as "a digital tool integrating various forms of documents and data (e.g. texts, tables, animations, pictures, sounds, video, etc.), which present and copy the reality to help and simplify the education."" [6].
According to Braga the Institute of Electrical and Electronics Engineers (IEEE) committee "defined a learning object as: "Any entity, digital or not, that can be used, re-used or referenced during technology-supported learning. "" [7].
This project using an application as a DLO, a valid option according to Braga. "Many applications can be used to support learning in a direct way and, therefore, can be considered learning objects. " [7].
One of the current trends and motivating factors of contemporary education is the implementation and use of information and communication technologies (ICT) in the educational process.
"Today, the teachers' approach to using ICT is more positive. From the perspective of a teacher this means a new way of teaching and from the perspective of a student a new way of learning. Learning through ICT becomes more interesting to students especially. Also building interdisciplinary relations and merging of various literacy automatically arises. Computers, educational software, digital learning objects, mobile devices, and interactive whiteboards also develop student creativity." [8].
In this way, it becomes evident the need for the development and improvement of digital media ICT tools, as defended by Kobs and Casagrande Jr. "[...] ICTs can contribute to education increasing the students involvement with learning objects from the use of smartphones and other technological instruments, both at school as out of it. " [9].
Tedesco is a supporter of the thought that "the incorporation of new technologies in education should be considered part of a global educational policy strategy" [10], and Harman and Koohang's article provides a brief theoretical review of DLO [11].
A historical review of Learning Objects (LO) from their initial analog conception can be found in work published by the MIT Media Laboratory [12].

B. Design and Production of DLOs
According to the research of Dias et al. [13], for a digital content to be considered a learning object, it must have characteristics that can be divided into two areas: pedagogical and technical.
"It should be noted that, as in any lesson plan, the proper selection of a learning object for use in didactic activity is defined based on the objective that is intended to be achieved in learning a given content. By looking at this question, the Learning Object can be an excellent ally of the teacher in the classroom." [14]. According to Tarouco [14] the project of constructing a learning object involves multidisciplinary skills, for this one can use the theoretical principles of instructional design of Gagné , Wager, Golas, & Keller [15].
"Authoring tools are essential resources for teachers to develop digital pedagogical content without the need to know a specific programming language. " [14].
"The evolution of authoring tools has contributed to a new scenario in which the production of digital educational material has been less and less restricted to the group of programming and design experts. Tools that provide the addition of interactivity and multimedia resources to digital content, without the need for programming, have provided the teacher with a new panorama, in which he sees himself not only as a user but also as a professional able to prepare their own Learning Objects." [14].
Another important reference material for the production of DLOs is the work of Rachel S. Smith [16] by The New Media Consortium (NMC) which offers practical advice for designing a DLO based on such concepts as usability, reuse, objects centered and oriented to student, current metadata standards, accessibility, and even marketing tips.
Finally, some of the existing DLO evaluation methods whose characteristics were studied for prototype creation were From this research on the different methods of design and evaluation of a DLO, 39 characteristics/attributes were defined to be used in the design and subsequent evaluation of the prototype. Of these, 17 related to fundamental design concepts and 22 features with pedagogical concepts for the assessment of DLO.
In the context of the bibliographical review, to point an experiment carried out in Brazil at one of the public federal universities (UFRGS -Federal University of Rio Grande do Sul). A learning object was developed for the teaching of cinema. The software was used as a support for Fundamentals of Cinema course, which was taught for three undergraduate degrees (Journalism, Publicity, and Propaganda). At the end of this work, they concluded that: "Perform an activity like this is not as simple as it first seemed. There are many unfolding and testing of an object in order to see the object as a whole function, both in terms of form and content. However, its development shows that it is possible to create interactive learning objects that use various information and resources available on the internet." [17].
The research reported a direct connection with the construction of learning objects for the teaching of theoretical contents related to the audiovisual area at a higher level. It can be shown as one of the closest experiments to what this project proposes, although the content taught is different. The lack of further research on the application of learning objects in the teaching of audiovisual production (at undergraduate level), corroborates with the fact that few teachers in higher education are qualified for this pedagogical reflection. The few publications on this subject evidenced the lack of formation or teacher pro-activity in search of better educational tools (analog or digital) in the audiovisual area.

C. Other Related Investigations
Other research that focuses on the use and evaluation of DLO can be taken as an example are the following.
The research on the influence of digital learning objects on concepts of zoology concepts in Portugal in which they considered "(...) possible positive influence of DLO on the educational process (...)" [18]. They also say that "(...) if teachers use the appropriate digital objects they can be helped in the teaching and learning process, because they will be inserting the theory with examples and language recognized by the students, in a more playful and illustrated." [18].
The use of learning objects for the development of skills among students with hearing impairment by Mahmoud [19]. The evaluation of didactic sequences and digital resources in the potentiating of the learning of algebraic concepts, where "The data indicate that it is feasible to use digital tools in combination with other resources that enable the student to establish the relationship between theory and practice." [20].
Bartek and Nocar's research represents the main features of DLOs used in mathematics teaching in the Czech Republic's primary schools [21].
Research has been found on the use of DLO for teaching computer programming in elementary school students [22]; in the Use of DLO in Religious Education [23]; the use of Digital Learning Material in Gymnasium Music Curriculum in Estonia for high school students aged 17-19 years "The preliminary results encourage the compilers that the created material is generally already well-produced and positively received by the students." [24].
Another collective effort to be scored is that found in the Nordplus Project: Networking, new skills and co-creation in Nordic Higher Education. Held between Nordic countries to insert emerging mixed education environments, developing specific DLOs for each discipline [25].
McGuinness and Fulton's research at University College Dublin, Ireland is also valuable as it aims to "to develop a suite of original interactive digital skills e-tutorials to be embedded in undergraduate and postgraduate courses" [26]. Apart from that "contributes useful insights to the body of literature on user engagement with digital learning objects in higher education, as well as students" perceptions and experience of blended learning" [26].

IV. METHODOLOGY
An application DLO has been created to make it easy to annotate relevant and essential data for the execution of visual effects (VFX) into audiovisual. In addition to providing fields for data collection, the application also provides some explanations of certain technical terms and links to external websites for a better understanding of certain concepts or to help in unit conversion calculations.
After using the application at the filming location, it makes available by email all the data collected in a completed form and forwards through a second email a script file (programming language of a computer graphic image software -CGI) created automatically by the application for use in CGI software. The script enables automating the task of reconstructing the data captured, directly in the CGI software.
After initial conceptual research, technical research and testing, prototype construction, testing and completion, field research began with 20 students and was completed with 14 students who effectively participated in a 60-hour course with 5 weekly meetings during 4 weeks, where they were subdivided into 4 groups. They performed 2 practical exercises that were divided into 2 different moments wherein each moment and exercise each group used or not the DLO to assist in the execution of the tasks.
At the end of the 2 practical exercises during the course students answered 3 forms so that they could evaluate the DLO in three dimensions: 1) product concept analysis, assessed primarily on a linear scale from zero to ten points by respondents (Likert Scale or Decis) [38]; 2) evaluation of the product characteristics regarding its relevance, which used the question format called Maximum Difference Scale [39] where it is possible to make a ranking of attributes to know which are considered the most and least important; 3) evaluation of product characteristics regarding its suitability, also used the MaxDiff question format.
Several methodological precautions were taken, such as the manipulation of an independent variable, forms of control and observation of effects, random distribution, among others. Works that served as the basis for this organization and methodological care for the development research part for consequent statistical and ethical validity were [40]- [43].
The research because involving the Portuguese research institution and being carried out in Brazilian territory is classified as international research by Brazilian institutions.
The Besides, because it involved human beings (students of undergraduate courses related to audiovisual production), it needed to be submitted to the Research Ethics Committee (CEP) and later to the National Research Ethics Council (CONEP) in Brazil.
The investigation received the CAAE case number: 03763418.6.0000.5188, it was appreciated in its methodology and ethical aspects and were approved for its execution, according to the presented planning.

A. Form -Product Concept
The first Form aimed at an analysis of the concept of the product, bringing the relevance, differentiation, credibility, and intention of use to be evaluated in a linear scale of zero to ten points by the respondents (Likert Scale or Dé cis) [38] besides other questions that used other scales for the answers.
For the representation of data analysis, it will be using the sum of the percentages of the first four items of the scale (Top 4: 7 to 10) and the sum of the percentages of the last four items of the scale (Bottom 4: 0 to 3).
In the four initial questions, the results were as follows: Question 01: The participants evaluated the tool with high relevance for students of courses related to audiovisual production. With a Top 4 response rate of 92.8%, no negative evaluations (Bottom 4 = 0%). Using the simple arithmetic mean of the assigned scores, we arrive at the 9.14 assessment on the scale of zero to ten (standard deviation of 1.23).
Question 02: In the differentiation question, the participants considered the tool highly differentiated from others they might have already had contact with. With a Top 4 response rate of 92.8%, no negative evaluations (Bottom 4 = 0%). Using the simple arithmetic mean the value assigned to this question is 8.85 on the scale of zero to ten (standard deviation of 1.29).
Question 03: The tool was evaluated with high credibility by the participants. With a percentage of Top 4 responses of 85.8%, without negative evaluations (Bottom 4 = 0%). Using the simple arithmetic mean of the assigned scores, we arrive at the 8.5 evaluation on the scale of zero to ten (standard deviation of 1.69).
Question 04: On average the participants evaluated their intention to use the tool in a high mode. With a percentage of Top 4 responses of 85.7%, without negative evaluations (Bottom 4 = 0%). Using the simple arithmetic mean the value assigned to this question was 7.92 on the scale of zero to ten (standard deviation of 1.73).
For questions 05 and 06 a scale of zero to ten points was also used, but the value of the value itself was not used, but a representation where zero has the meaning of "totally disagree" and in the other opposite of the scale the value ten has the meaning of "I totally agree".
Question 05: "Learning, having competence and qualification to produce more elaborate visual effects (integration between real and virtual images in audiovisual) can improve my possibilities of insertion in the labor market". The participants agreed very strongly with the statement. With a Top 4 response rate of 99.9%, no negative evaluations (Bottom 4 = 0%). Using the simple arithmetic mean of the assigned scores, we arrive at the 9.57 assessment on the scale of zero to ten (standard deviation of 0.93).
Question 06: "Learning, competence, and qualification to produce more elaborate visual effects (integration between real and virtual images in audiovisual) allow me to have greater creative freedom in my audiovisual projects". The participants agreed very strongly with the statement. With a Top 4 response rate of 99.9%, no negative evaluations (Bottom 4 = 0%). Using the simple arithmetic mean of the assigned scores, we arrive at the 9.57 assessment on the scale of zero to ten (standard deviation of 0.93).
For question 07 a linear scale of zero to ten points was also used, however, the assignment of the value itself was not used, but a representation where zero has the meaning of "would not recommend with certainty" and in the other opposite of the scale the value ten has the meaning of "would recommend with certainty".
Question 07: Participants would highly recommend the tool to a colleague, relative or friend. With a Top 4 response rate of 92.8%, no negative evaluations (Bottom 4 = 0%). Using the simple arithmetic mean of the assigned scores, we arrive at the 9.0 assessment on the scale of zero to ten (standard deviation of 1.3). This question, however, is based on the Net Promoter Score (NPS) methodology used to measure public satisfaction and aims to assess how well the company or product is recommended by the so-called definitive question.
As the NPS value computed in the question was 64%, by the standard scale we can verify that the index is within the Quality Zone, something very good since it is not classified in the Critical Zone or in the Improvement Zone (where it would require interventions and adjustments, deeper and/or structural adjustments). In addition, it is also perceived that there is scope for the product to develop, in order to seek to fit into the Zone of Excellence in the future, using the information gathered in the next Forms for its improvement.
Question 08 of this first form was the only one that did not use a linear scale from zero to ten. When requesting a comparison between the concept seen from the created tool and what is currently on the market, a linear scale of one to five was used where one could choose between comparative statements.
In this last question of this first Form, no participant understood that the product (tool) is worse or much worse than what already exists today. Two participants 14.3% answered who is neither worse nor better than what already exists today, the rest, 85.7% answered that they understand that the concept of the product they saw is better (71.4%) or much better (14.3% %) than what already exists today.

B. Form -Relevance
To create this Form of relevance in the MaxDiff model, all 39 attributes created were used. All attributes were randomized by the MaxDiff survey application that was configurated to create 30 comparative questions with 4 attributes each question (application makes the statistical distribution of attributes to the questions). Students had to mark in each of the 30 questions on the form, which of the 4 features he considered the most and least relevant. The 30 questions were randomly presented to each student participating in the survey to avoid bias related to tiredness if it had a single order for presentation the questions. As a result, it was verified that the attributes selected as the most relevant attributes of the tool were: Total average relevance Efficiency -a perception of being competent, productive, of achieving the best yield with the minimum of errors and/or expenditures.

98.3
Technical functionality -if it fulfills its purposes: assistance in data collection and automation in the transfer of these to CGI software.

96.0
Content quality -concepts, information, references, images, etc. used in the tool (reinforce key points and significant ideas).

75.6
Utility -a perception that the use of the tool is valid.
69.0 Coherence -logic, meaning between the contents, the objectives, the activities developed, the evaluation and the profile of the student.

68.5
In this Form, in relation to statistical metrics, the mean value of the attributes was 42.7, the standard deviation was 25.4 and the adjustment, measured by the RLH (Root Likelihood) was 0.6962, considered to be good (this parameter varies between 0 and 1, the closer to 1, the better the adjustment -values above 0.6 are good). According to the calculations, the attributes to be valued were those that obtained in their total average of adequacy value above 68.1.
The average value of importance 42 (on a scale of 0-100) is due to the fact that there are attributes that are of little importance to the respondents, and that, therefore, have amounts close to 0, which translates into a high standard deviation (also due to the number of subjects).

C. Form -Adequacy
To create this Form of adequacy in the MaxDiff model, 16 attributes were selected among the 39 attributes created for the relevance Form. The ones that most appear in the various methods of evaluation of DLOs studied. These attributes were randomized by the MaxDiff survey application that was configurated to create 12 comparative questions with 4 attributes each question. Students had to mark in each of the 12 questions on the form, which of the 4 features he considered the most and least relevant. The 12 questions were randomly presented to each student participating in the survey to avoid bias related to tiredness if it had a single order for presentation the questions.
Although it was not possible to reach a larger number of subjects, which limits the analytical robustness, the research model presented good results. It was verified that the attributes selected as the most adequacy attributes of the tool were: Total average adequacy Technical functionality -if it fulfills its purposes: assistance in data collection and automation in the transfer of these to CGI software.

72.1
Pedagogical objectives -identifiable and appropriate to the target audience. Assistance in the practical activities of audiovisual production with real x virtual interaction.

61.4
Utility -a perception that the use of the tool is valid.
50.0 Help in learning -provided by the tool as an educational resource (learning object).

49.1
In this Form, in relation to statistical metrics, 21.1 values were obtained, the standard deviation was 23.9 and the adjustment, measured by RLH (Root Likelihood), was 0.7630, considered to be good (this parameter varies between 0 and 1, the closer to 1, the better the adjustment -values above 0.6 are good). According to the calculations, the attributes to be valued were those that obtained in their total average of adequacy value above 45.
The average value of importance 21 (on a scale of 0-100) is due to the fact that there are attributes that have aspects that are not very suitable for the respondents, and therefore have amounts close to 0, which translates into a high standard deviation (also due to the number of subjects).

D. Crossing between Relevance and Adequacy
Crossing the data from the importance/relevance Form with the adequacy Form, it turns out that the tool has adequacy below the relevance in the most important attributes, which means that it must be improved. With the crossing, it is possible to infer that the attributes "Technical functionality", "Utility" and "Content quality" present statistically significant value for greater relevance than adequacy.
Finally, although the adequacy analysis has shown relatively low values, there are many aspects that can be improved from the relevance analysis. Effectively, by improving aspects with amounts over 60, for example, it may be possible to build a new version of the tool even better than the current one.

E. Development Research -Blind Analysis
At the end of the mini course, each participant delivered in a digital format their practical exercises developed during the course. These files were renamed and identified only by numbers, and all kinds of information (such as metadata) were deleted from the files so that authors could not be identified.
These studies were submitted for evaluation by 3 external evaluators to this research, two of them being undergraduate Brazilian professors responsible for disciplines related to audiovisual production, and an evaluator who, although not a teacher, works in the audiovisual area in the city of Lisbon, Portugal.
To grade the works, audiovisual technical criteria were used as lighting elements (color temperature, light type, light power, positioning, shadows), animation, movements, clipping masks, chroma-key quality. Also, with composition criteria such as the soundtrack, sound effects, montage. The main orientation was the degree of correspondence and verisimilitude between the images generated by computer graphics integrate with the real ones and the quality level of this integration.
After collecting the values (notes) attributed to the practical work by the three evaluators, no differences were observed between the use and non-use of the tool in the quality of the work developed. Although the statistical calculations also present this result, the simple arithmetic mean of the evaluations of the three evaluators allows inferring the non-significant difference between the situation between use or non-use in the final audiovisual product result.

A. Impact on Society
Online learning tools, such as DLOs, allow students different ways to interactively acquire skills and content. In addition to reusable DLOs (especially if hosted in repositories), DLOs can be accessed at any time you need them, both while performing classroom tasks, with as teacher support and guidance, and can review and reinforce the learning of the treated content.
It is also important to consider was not found using the search key terms most commonly related to DLOs production, results about the learning experience or DLOs designed to assist in teaching audiovisual visual effects production (VFX) for undergraduate education. This fact highlights part of the impact that the DLO created can aggregate the area in which it is inserted.
Besides, it did not meet either any software for personal computers or mobile applications with the properties and automated functions found in the developed application. In this way, the research also gives a valuable contribution and increasing the possibilities available to aid in the production of VFX in audiovisual productions.

B. Recommendations
Instructors are encouraged to strategically incorporate digital learning objects into their courses to assist and reinforce classroom learning as well as to support the development of specific skills. Guidelines to ensure ease of access to DLO, continuous user experience and timely feedback are to be observed and provide adequate support for rapid resolution of technical failures (especially in DLO prototypes under test and evaluation).
The differences found in the comparison between the concept, relevance and adequacy Forms (all positive) when compared with the blind analysis results (which did not show qualitative gain or decrease in the results), indicate the future recommendation of attention to the methods, to demonstrate clearly that the Hawthorne effect does not influence the results.
Moreover, DLO show to be a valid tool for students to learn in the digital environment. However, further exploration of collaborative studies is needed to provide comparative studies that go beyond case studies.

VII. CONCLUSION
By highlighting which characteristics and attributes were assessed as most important/relevant in the adequacy and relevance analysis, cross-checking the conceptual and blind analysis allowed the case study to contribute useful information to the scope of the literature on the actual contributions of digital learning objects for teaching as well as with students due to their perceptions and experiences with blended learning.
The data obtained allow us to verify that the prototype developed successfully achieved its main objectives, as it fulfilled its technical purposes of assistance in data collection and in the automation task that transfer of these to CGI software, and may well be used outside the educational context. Also, with an indication of use by the participants. Likewise, it achieved its pedagogical objectives in helping in practical audiovisual production activities with real vs. virtual interaction.
The development of the research (which is part of doctoral research) may with articles and the thesis itself, point out how the experience was enriching and valid in the pedagogical context. Besides, it can serve as a reference for other teachers to develop their own DLOs, having theoretical references based on the pedagogical and design areas.
The research contributed both the experience of creating a DLO that was designed and tested and evaluated in the classroom, as well as submitting the tool to two international repositories of digital learning objects, Merlot Repository [44] and OER COMMONS. -Open Educational Resources, which after peer review have included DLO in their respective catalogs and are now available for access by any individual (teacher, student or even practitioner).
The availability in repositories is an important feature in the definition and conceptualization of a DLO according to several researchers in the area [4], [7], [14], [34], [36], [37]. The possibility of sharing allows it to be accessed and used by other teachers. Thus, the choice of the English language for the tool has allowed it to be disseminated to be used by a much larger number of international institutions.
It is also believed that the work reached a conclusion similar to that other works such as Popovich that conclusion that "Educators can add this study to the growing body of research regarding the effectiveness of digital learning objects and other open education resources as effective learning supplements. (…) that digital learning objects can be employed to aid in student learning, (…)" [45].
Just like the Research conducted with medical students for radiology education that concluded that "The blended learning method has a significant impact on performance during testing compared to the traditional method. The implementation of DLOs that complement face-to-face education makes it possible to strengthen the teaching process with high levels of satisfaction, justifying the time and resources required for their design and production." [46].

CONFLICT OF INTEREST
The authors declare no conflict of interest.

AUTHOR CONTRIBUTIONS
AM conducted the research, analyzed the data and wrote the paper; NC was the advisor of the research; all authors had approved the final version.