Flipped Classrooms for Introductory Computer Programming Courses

Teaching and learning how to program are not easy tasks. Disapproval and dropout rates are a concern for everyone concerned with the topic. Therefore, it is necessary to look for strategies that improve the motivation of students who start a programming course, also improving success rates and decreasing dropout rates. The inverted class model, or flipped classroom, has been used in several experiments, showing very good results. The objectives of this teaching-learning technique is to change the traditional order: students have contact with new subjects before the classroom, using videos, texts or other material, as well as small online tests to check their knowledge. In this way, the face-to-face classes are reserved for discussion, doubts and application of previously acquired knowledge. The purpose of this paper is to analyze the scientific production on Flipped classrooms in introductory programming courses indexed in Elsevier’s Scopus. The sample is composed by 45 articles in total. The results obtained by bibliometric analysis showed when and where those documents are published, who are the authors and what is the focus of said articles. We also analyzed the most cited documents. We made a summary of the articles, namely in what refers to the sample size of the experiences, which programming language is used, in which universities the articles are made, which technology is used, as well as which methods are used in order to create inverted classes and which are the objectives and results of these experiences reported on the articles.. We managed to get a global view of the theme, getting a strong analysis for those who want to use flipped classrooms for teaching programming.


I. INTRODUCTION
Flipped classroom is a strategy that has been widely used in different teaching-learning subjects at the university. The dynamics of this teaching-learning technique is to invert the traditional order: students have contact with new subjects before the classroom, using videos, texts or other material, as well as small online tests to check their knowledge. In this way, the face-to-face classes are reserved for discussion, doubts and application of previously acquired knowledge. There are great advantages: each student learns at his own pace and while during classes and interactions with the teacher he can make better use of time, achieving an application of the knowledge he previously acquired. In this way, the instructors can propose dynamic classes and solve more interesting exercises, not wasting time with classes to transmit knowledge. The concept of flipped classrooms intends to invert the way content is taught and applied. If, in Manuscript received September 11, 2020; revised December 24, 2020. Sónia Rolland Sobral is with REMIT, Universidade Portucalense, Porto, Portugal (e-mail: sonia@upt.pt). the traditional format, the contents are usually presented in a face-to-face class and later applied to homework, in an inverted format it works the other way around: the contents are learned by the students before classes and the application of knowledge is done in the classroom with the accompaniment of the instructor. There are several definitions: "educational technique that consists of two parts: interactive group learning activities inside the classroom, and direct computer-based individual instruction outside the classroom" [1] or "specific type of blended learning design that uses technology to move lectures outside the classroom and uses learning activities to move practice with concepts inside the classroom" [2]. Not everything is easy in this paradigm: the materials for the students have to be very efficient, the students have to do their previous work and the instructor has to be able to understand what is the best support he can give to his students. The responsibility for learning is on the student's side so students can work towards mastery of the material [3]. This means, in Bloom's revised taxonomy, that students do the lower levels of cognitive work outside of class and focus on the higher forms of cognitive work in class [4]. And are all students capable of doing their homework before classes? Can this technique be used by less motivated students? And as for the students that are highly motivated: do they have advantages in being guided by this type of inverted classes? Despite the doubts, several reports have been made to show that the adoption of inverted classes can have good results.
Teaching and learning how to program is not an easy task. Disapproval and dropout rates are a worrisome matter for everyone concerned with the topic. Therefore, it is necessary to look for strategies that improve the motivation of students who start a programming course, also improving success rates and decreasing dropout rates. The inverted class model, or flipped classroom, has been used in several experiments, showing very good results. This pedagogical approach has been adopted by several institutions, by engineering and science courses. There are some reports of the use of flipped classrooms in teaching programming, which can be a good alternative to traditional methods. And the goal of an interested teacher is the constant search for different approaches to teaching and learning.
The purpose of this paper is to analyse the scientific production on Flipped classroom in introductory programming courses indexed in Elsevier's Scopus. The sample is composed by 45 articles in total. The results obtained by bibliometric analysis showed when and where those documents are published, who are the authors and what is the focus of said articles. We also analysed the most cited documents. We made a summary of the articles, namely in Flipped Classrooms for Introductory Computer Programming Courses

Sónia Rolland Sobral
International Journal of Information and Education Technology, Vol. 11, No. 4, April 2021 what refers to the sample size of the experiences, which programming language is used, in which universities the articles are made, which technology is used, as well as the methods to create inverted classes and which are the objectives and results of these experiences reported in the articles. We managed to have a global view of the theme, getting a strong analysis for those who want to use flipped classrooms for teaching programming. This article is subdivided into several sections: the research questions, the methodology, the bibliometric results, the results of article content, ending with the discussion of results, the conclusions and future work.

II. THE RESEARCH QUESTIONS
To determine how data is identified, collected and presented, research questions are defined [5]. We can divide our research questions into two types: regarding bibliometrics and regarding the content of publications. The purpose of this work includes several research questions, namely: Bibliometric questions: How has the evolution of the publications related to flipped classrooms for introductory computer programming courses been? When were the articles published?
What is the type of these publications? Where were the articles published?
What is the focus of flipped classrooms for CS1? Who publishes on the subject? Are there clusters of authors who publish together?
What are the most cited articles? Content related questions: What are the characteristics of the studies presented in the articles?
What is the number of students considered in samples? What programming language is used? Which universities are identified in these studies? What technology was used? How were the inverted classes put into practice? What are the objectives of the publications? What are the results found in the studies identified in the research?

III. METHODOLOGY
In 1969, Alan Pritchard [6] used the term bibliometrics that came to be used in information science. Bibliometric analyses of scholarly literature are used for a deeper understanding of scholarly activity and performance and highlight their policy relevance [7]. In scientific research, it is important to get a wider perspective of research already being conducted concerning a relevant subject matter [8] and a bibliometric analysis profile on the research trajectory and dynamics of the research activities across the globe [9]. This study systematically analyses the literature using articles indexed at Elsevier's Scopus (Scopus). Our aim is conducting a bibliometric and literature analysis of papers that we expect provides a useful reference for future research. The search strategy was TITLE-ABS-KEY ( "Flipped class" OR "inverted class" OR "Flipped classroom" OR "inverted classroom" AND ( cs1 OR "CS 1" OR "introductory programming" OR "introduction to programming" OR "novice programming" OR "novice programmers" ) ) AND PUBYEAR < 2020.

E. The Ten Most Cited Documents
Nine of the ten most cited documents are conference proceedings and the other one is a journal paper [10]. The universities of the ten most cited articles are Arizona State University College of Technology [11], University of Helsinki. [12], University of North Carolina. [13], CSU Monterey Bay [14], University of Toronto [15]- [17], University of Pittsburgh [10] and Trinity University [18]. One of the articles compares ten universities [19]. The first [11] was cited 67 times and the 10th [19] was cited 11 times.

A. Number of Students in the Experiments
The average number of students in each of the experiments was 391. The article that reports the least students was [20] and the one that uses the most students is [10].

B. Programming Languages
There are several programming languages that are used in articles: Java appears the most.

D. Purpose
One of the articles reviews the subject [43], most articles compare the use of inverted classes with traditional classes [16], [29], [30]. Different objectives are referred, as programming self-efficacy and academic performance [25], gauge the use of the MUSIC model [36] or check for different acceptance by gender [42]. Two documents [31] use two different approaches: students bring computers to class and the other approach is "to ban computers from the classroom and to require students to solve programming problems on paper".

F. How
Most teaching-learning dynamics go through online versions of the course textbook [28] and videos before laboratory classes: 22 video tutorials [11], seventy-five 5-15-minute videos [15] or 7-11 minutes [11], one hour of video every week to prepare for lecture [16]; Videos are usually created by the instructor or chosen by him, but there are cases where students are encouraged to look for videos that fit the given subject [27]. Some articles report multiple-choice quizzes over the video material [18] or at the beginning of classes [24]. Inverted classes are generally used as a teaching-learning strategy, but in some cases Partially Flipped Classroom Model is used [21].
As an example of flipped class dynamics: "In class • Instructor revises goals and objectives of the class (5 mins) • Students ask questions and clarifications ( answer a short quiz (10 mins) • Instructor specifies goals and objectives for the next class (5 mins). Out class: • Students watch videos • Read lecture notes / reference material • Solve homework problems." [25].

G. Findings
Some findings: Viewing long static videos can be boring and course overwhelming [11]. Students generally perceive the online lectures as helpful [34]. Intrinsic motivation did not impact performance in the flipped classroom [37]. Self study gives more motivation to study programming and it is appreciated more by the students who like self-pace [48]. Students prefer inverted classroom rather than traditional lectures. 60% [14]. Students' enthusiasm for the course increased over the semester [15]. Drop, Pass and Failure Rates compare to traditional lectures [15], [17], but significantly higher grades on the exam are the inverted offering compared to the traditional offering [16], as the inverted offering encourages students to "go big or go home" [17]. 57% of respondents preferred using class time for problem solving or active learning with the instructor present versus listening to a lecture [11].
Gate-check video quizzes are not sufficient to improve the grades of students in a flipped CS1 classroom [18] or flipped classroom results in enhanced student performance in the final exam and also contributes to improving competency acquisition [25], [30].
By adding explicit learning objectives and exercises to be completed before students come to class, these worksheets can improve student preparedness, allowing for class-time to focus more on advanced learning objectives [20].

VI. DISCUSSION AND CONCLUSIONS
The purpose of this work includes several research questions, namely, as bibliometric questions: How has the evolution of the publications related to flipped classrooms for introductory computer programming courses been? When were the articles published? The 45 articles are from 2012 to 2019. From the year 2015, inclusively that year,, the number of articles varied between 7 and 9.
What is the type of these publications? Where were the articles published? 76% of the articles are conference proceedings. Nine presented at the ACM Technical Symposium On Computer Science Education and eight presented at the Frontiers In Education Conference.
What is the focus of flipped classrooms for CS1? By the keywords we can see that the focus of the articles is teaching (7%), student (6%), and Flipped Classroom and Education Computing (5% each).
Who publishes on the subject? Are there clusters of authors who publish together? Michelle Craig and Diane Horton, both from University of Toronto (Canada) are the authors with the most articles (three each). There are five clusters: three are from the same university (University of Toronto, Karadeniz Teknik Üniversitesi and University of Calgary) and two have researchers from more than one university.
(Brandeis University, United States plus King Saud bin Abdulaziz University for Health Sciences, Saudi Arabia and University of Houston plus Trinity University, both from the United States). What are the most cited articles? The most cited article [11] was cited 67 times. University of Toronto is the affiliation of three of the ten most cited articles in the field.
Regarding content issues: What are the characteristics of the studies presented in the articles?
Most articles report experiences of using inverted classes to teach programming. Only one of the articles compares the experiences of ten universities and another article presents a literature review.
What is the number of students? Most of the article's present experiences with less than 100 students. What programming language is used? The Java programming language is the one that appears most often in the 45 articles.
Which universities are identified in these studies? There are reports of units from twenty-five institutions. What technology was used? There are different types of technology used, but most use YouTube videos, Facebook groups and learning management systems (MOODLE or blackboard).
How were the inverted classes put into practice? Most experiments use videos (short) and quizzes about these videos to be made before lectures. In face-to-face classes, the instructor provides support to students and promotes exercises to apply the knowledge obtained before classes. There are many forms of work: using pair programming, project-based teaching, for example.
What are the objectives of the publications? Most articles compare the use of inverted classes with traditional classes. The objective is to verify if classes with the Flipped classroom paradigm are more efficient (in different aspects) than traditional classes.
What are the results found in the studies identified in the research? There are several types of conclusions: it is generally reported that students liked the experience, although students find it boring to watch videos before classes and others do not want to interact on Facebook. If some experiences reflect an improvement in students' skills, others say that inverted classes may not be enough for all types of students.
Our conclusion is that this is an open hypothesis for teaching programming to university freshmen. The methodology seems very interesting for very motivated students, but it can have bad results for students who don't like programming and learning how to program so much. These students may find it difficult to do their homework prior to classes and thus go (or stop going) to lectures and not understanding anything at all. The quality of the videos can be a problem: there are several teachers who report that they took a long time to make the videos and that they had no skills on how to make them (sound quality, dynamism, pedagogical strategies); however, looking for links from YouTube videos may not be a good alternative. It seems that this is a dynamic that makes good students better and bad students (or unmotivated or with less skills for programming) worse. These are the questions that will have to be analysed