Improving the Teaching of ICT Engineering using Flipped Learning: a personalized model and a case study

Abstract

In recent years, Flipped Classroom started to be used as an effective way of teaching Engineering among various strategies in higher education. However, enabling and using the flipped learning is a complicated task, not a straightforward goal that can be simply achieved through a combination of face-to-face and online activities. It requires a more sophisticated understanding of effective teaching methods to manage the shift from the traditional to the flipped learning and the optimum adaptation of technology as part of this change. Given this challenge, this research work provides a personalized model of the flipped classroom and investigates through a case study in an Engineering School how our approach can be used to improve teaching of Information and Communication Technology (ICT) Engineering. It assesses by using empirically data related to the interaction of the various actors at different levels of abstraction, particularly from a gender perspective, the relevance and the impact of the flipped classroom on student learning and achievement in ICT Engineering Education.

Keywords
Flipped classroom; Engineering education; ICT; Gender sensitive teaching

1. Motivation and objectives

Given the increasing number of students and the limited staff and resources in the developing countries, their universities face a lot of difficulties in the last years: (1) the predominance of a theoretical and general education as compared to the real business/economy needs, (2) the low employability of graduates, (3) a traditional teaching approach based on teacher-centric classrooms with a limited use of Information and Communication Technologies (ICT) in the educational process, (4) low communication skills of the students and a lack of collaboration and team learning. To meet the targets of the education at universities under such constraints, engineering schools were created for better professional career preparation according to the new job demand. In this case, one of the first leading ideas was to renovate training, improve the face-to-face teaching, and build hybrid and attractive learning situations to support technology-driven classrooms. Usually, the traditional teaching consists of two successive major phases: 1) a knowledge transfer where the student has to listen passively to a lecture (in-class phase); and 2) a knowledge use that takes place mostly outside of the school according to the teacher directions and a given homework (outside phase). An alternative learning mode called “flipped classroom” that reverses these two phases and includes active learning (Web Based Training, PBL, projects in groups, etc) has become popular recently. Inspired by the works of Dewey, Montessori, Bloom and Freinet, the flipped classroom has been highly practiced in association with Peer Instruction in the previous work of Mazur (1997) and recently studied and highlighted in the works of Strayer (2012) and Faillet (2014) to name only a few. This blended method of learning takes benefits from the advantages and the strengths offered by both distant and face-to-face learning (Bissonnette & Gauthier, 2012). Thus, it appears that the flipped classroom is supposed to be a good solution for education in engineering and will continue to grow in popularity and success in the near future (Bishop & Verleger, 2013). Hence a useful direction of reforming education in our engineering school could be to experiment the flipped classroom for our students who are candidates to obtain a professional degree of engineering. Through such experimentation, we aim first to evaluate the impact of the flipped classroom in our context/environment and to understand by the way the difficulties and challenges that could face the students during such mode of learning change and shift of paradigm. The second objective is to specify a concrete approach to support the change from the traditional to the flipped learning and achieve the expected improvement in our engineering education system.

So far, our research is still a local initiative. However, once the experience of the flipped classroom is finished with good and proven results, it will be easy to convince our colleagues, other teaching teams and our school administrators to deploy it widely in an organized, strategic and sustainable form. Thus, this paper aims to evaluate the flipped classroom from the students point of view in the context of ICT engineering education and to quantify how relevant is the flipped learning in improving the performance of these students during the 4^th year of Engineering in Information System and Communication (EISC) curriculum (i.e. a 5-Year engineering degree (Master's degree)). This experience is executed by performing the following specific tasks:

• ●

  Comparison of a usual in-class training scenario with a flipped classroom at the instructional design level in order to develop a personalized model adapted to ICT engineering education;

• ●

  Analysis and assessment of the learning process and the student’s behaviour during the flipped classroom;

• ●

  And measure of performance and satisfaction of the students in a flipped classroom, particularly during collaborative trainings and from a gender perspective.

In fact, the main scientific motivation of this study is to build after all a value-added expertise to help colleagues working in the field to make an easy move toward setting up flipped classrooms in ICT engineering education by the optimal use of available human and organisational resources.

The rest of the paper is structured as follows: section 2 presents the related work on the flipped classroom. The theoretical and pedagogical approaches adopted during our experience of the flipped classroom are also discussed in this section. The third section describes the implementation of the experiment that we conducted in the ICT engineering education for the EISC branch. The last section provides discussions and a summary based on the analysis of the results we have obtained in reference to our objectives. Finally, in conclusion, we discuss key points of our approach and our perspectives.

2. State-of-the-art and theoretical basis

2.1. The flipped classroom approach

The origin of the flipped classroom pedagogy is from the work of Jonathan Bergmann and Aaron Sams made in the mid-2000s and was subsequently heavily popularized by OpenCourse-Ware (OCW) of the Massachusetts Institute of Technology (MIT) and Khan Academy founded by Salman Khan and MIT, who proposed the use of educational video resources to “reverse” classes. Inspired by these initiatives, the professors Sebastian Thrun and Andrew Ng teachers initiated an academic partnership (Coursera partnership) between several universities to launch large multimedia-based education courses (Bissonnette & Gauthier, 2012). Meanwhile, MIT and Harvard University have launched also “EDX”. Such initiatives have put a pressure on universities to offer their courses with similar flexibility and improvement of performance in parallel of the face-to-face context and according to the recommendations of the International Standardization Organizations. For example, ABET (Accreditation Board for Engineering and Technology) has issued the specifications to be respected and the capacities to be developed over a technical engineering training; namely (Accreditation Board for Engineering and Technology, 2013):

• ●

  Skills for efficient communication;

• ●

  The ability to identify, formulate and solve engineering problems;

• ●

  The ability to design a system or process under real professional constraints (technical, economic, social, environmental, etc.);

• ●

  The ability to work within a multidisciplinary team;

• ●

  The ability to use ICT tools to solve engineering problems.

Consequently, it will be difficult for students enrolled in an engineering training to develop these skills through conventional classes without using new teaching models (Felder & Brent, 2003; Diefes-Dux et al., 2006; Yildirim et al., 2010). Nowadays, the flipped classroom is the buzzword of the pedagogy and education literature (Roberge, 2016). However, there is no recognized and unique model of this form of education. A simple definition was given by (Lage et al., 2000, p. 32): “Inverting the classroom means that events that have traditionally taken place inside the classroom now take place outside the classroom and vice versa”. While Bishop & Verleger (2013, p. 5) provide a more broad definition that we have adopted in our research because it is a priori valid for the ICT engineering education as it was for several scenarios which have been proposed in different research works: “The flipped classroom is an educational technique that consists of two parts: interactive group learning activities inside the classroom, and direct computer-based individual instruction outside the classroom.”

Two main questions must be answered: a) Is it efficient to include flipped learning as part of university education and higher education (e.g. engineering education)?; and b) Are there any answers from previous research works that has been based on empirical data with clear and scientific evidence? By trying to answer these two questions, we found that Bissonnette & Gauthier (2012) conducted a systematic review of the scientific literature published in peer-reviewed journals and indexed in the main education databases between 2005 and 2013. From their point of view, studies to take into consideration as relevant research should meet the following criteria:

1. 1

   must have evaluated the impact of flipped learning on education, more specifically on students’ performance;

2. 2

   must have used an experimental or quasi-experimental research protocol using a comparative reference group;

3. 3

   must have been published in a peer-reviewed indexed scientific journal.

A first exploration of the literature allowed identifying a significant number of research works published in this field (33 articles in the ERIC database and 223 articles in PsycINFO). Surprisingly, they found only Pierce & Fox (2012) paper which seems to fulfil really all the three inclusion criteria set above. Although the work results have shown a better performance in the flipped classroom (an increase of 3.9% between the final marks in a flipped classroom in 2012 compared to a traditional course in 2011), this study does not provide enough information about the equivalence of the two student groups involved in the experiments which represents, in fact, a significant limitation of this research and the interpretation of its results. As far as we are concerned, we found a more recent study of a flipped classroom of physics and chemistry courses at a high school (Faillet, 2014) that fulfils also the three criteria of Bissonnette & Gauthier (2012). This study showed that students with high level performance in the traditional class are generally less efficient in the flipped classroom, while students with the lowest performance in the traditional classroom are more efficient in the flipped classroom.

Our present study is a continuation of the previous research, including works of Bissonnette & Gauthier (2012), Faillet (2014) and some very recent research publications (See for instance Lucke et al., 2016; Chiang & Wang, 2015). Furthermore, the particularity of our research, aside from the practice of a flipped classroom in a real context according to the three criteria, it focuses on higher education (ICT engineering and Master's degrees) in an engineering school and university.

2.2. Theoretical foundations and pedagogical approaches for experimental purposes

In order to get feedback of experience and to assess in detail the benefit and the efficiency of the flipped classroom for engineering, it is important to develop an approach that is adapted to our reality and meets the needs of both students and teacher. However, the integration of the flipped classroom into engineering training is not a simple task because, from our point of view, the whole education system will be affected (i.e. in-class and out-class learning activities). In fact, integrating the flipped mode in higher education involves a rethinking of the pedagogy, approaches, methods of intervention, spaces, places and time scheduling, values, behaviours, representations and practices of the different actors in the education system. Therefore, we have started by a design of an experiment model and a social environment that provide all the components necessary for the efficient execution of a flipped classroom. To fulfil our experimental purposes, we have chosen several educational models that led to the development of a personalized and blended model combining in-class and out-class activities supported by the ICT. As with all our requirements, we have chosen the active learning model as an overall teaching approach. It offers various opportunities for social interaction and learning so it could lead when adequately employed to individual knowledge construction and social co-construction of knowledge in/out of the classroom.

2.2.1. Active learning and Project-based Learning

Active learning is well suited to our training objective. This is a very relevant choice for our targeted training, where the focus is on engineering education, learners are adults, and the areas of expertise are related to ICT (i.e. networking and computer science engineering). Our goal is to create an environment where students can learn how to cope with difficulties and complex problems encountered in the domain of computers and modern networks, by discussing, integrating and collaborating in a team/group to reach a deep level of understanding. We expect that in a team the student will reach a high performance that he/she could never achieve alone. Indeed, in the EISC curriculum, the theoretical knowledge is not sufficient to acquire a satisfied level of the required skills. This is why we present a case study enabling our students to think and work together to find solutions to concrete problems. In this perspective, we have established an instructional design, which makes from the students more active and go beyond the stage of receiving and understanding information to a more advanced stage of analysis, synthesis and creative confrontation with the other student's knowledge. In addition, students must gradually become very conscious of the immense richness of information given by the Internet; in fact, one can find very interesting content through the well-targeted web search. The tools to find, to access and to categorize this information are becoming more and more efficient. However, such information can contribute to learning only if it is channelized through meaningful activities, carrying high-level skills, geared towards the achievement of well-defined and adapted goals. This is one of the objectives of the required model as part of our overall flipped classroom approach. Hence, we invited students to accomplish in-class and out-class activities individually and in a small team and in a given time period under the teacher's assistance in order to practice active learning through a global strategy. In the next section we present the model “Search, Structuring and Confrontation” (SSC) as a model we adopted and personalized to support the flipped learning and the active project-based pedagogy.

2.2.2. The Model “Search, Structuring and Confrontation” (SSC)

The SSC model (in French RSC: Recherche, Structuration et Confrontation) (Betbeder et al., 2006 and previous conferences work of the same authors) is an active learning model which is based on three major steps: (1) Search of information, then (2) Structuring of information and finally (3) Confrontation and collectively debate on the structured information. The first two steps (Search and Structuring) are out-class activities (web search, asynchronous web-based video lectures, e-brainstorming, individual understanding of ideas, debate and team discussions, and team production of documents). The phase of Confrontation is a collective classroom training stage, which can alternate sessions of presentations and practical exercises/labs. This sequencing is pedagogically planned to encourage students to (1) present in class structured knowledge and (2) to provide a comparative analysis of IT engineering market solutions. In this case, students will be able to identify the correct criteria for selecting a very specific IT solution and develop skills/knowledge to present and defend it in front of an interested professional audience. From our point of view, although the SSC was used by Betbeder et al. (2006) only to enhance collaborative Web Based Training (WBT) not blended or in-class learning; but it is still very useful in our case. First, this model of 3-steps is comprehensive and easy to implement in ICT context. Secondly, it matches well with the different phases of the flipped classroom. It will allow the teacher to model and manage effortlessly the in-class and out-class activities so that each student will be engaged to carry out the proposed task and achieve the common goal. Finally, the SSC will also give some dynamic to this activity and develop a team-spirit in the class.

2.2.3. The size and the composition of teams for the SSC model

In the majority of research on collaborative and active learning, the team size is generally from 2 to 5 learners (Depover et al., 2004; Stoyanova & Kommers, 2002; Van Boxtal & Veerman, 2001). Recommendations for the engineering training suggest using teams of three students (Accreditation Board for Engineering and Technology, 2013; Felder & Brent, 2003; Diefes-Dux et al., 2006). Moreover, the team composition can be done in several ways (i.e. in random, pragmatic, progressive or opportunistic manners). Some authors like Light & Mevarech (1992) emphasize the importance of the conflict and open debate as a driving force and a key factor to enhance information exchange within a team, while other authors believe that collaboration can also take place even in didactic situations where certain learner’s disagreement could happen (Dillenbourg et al., 1996).

Other papers (Depover et al., 2004; Benayed & Verreman, 2006) highlight the importance of team size and the choice of the learning period and recommend activities of a small team over time periods sufficiently significant to get a good result from the collaborative activities. Any way, it seems that different variables related to the constitution of the groups, such as conflict creation, confrontation of ideas and disagreement between the learners could influence the interactions inside the team framework and, to some extent, influence the entire group productivity. Thus, we recommended to our students to build teams of three persons. But, we did not restrict the choice of the team's partners since our context is about education in engineering. Because we were interested in a natural team’s composition, we have encouraged our students to formulate openly their wishes about the choice of their teammates.

3. Research methodology and description of the experiment

3.1. The choice of the context of the flipped classroom

After fixing the basic choices related to the pedagogical scenario, we designed an experiment based on the recommendations provided by researchers who have worked on the flipped classroom. As a result, we have chosen the module “Networks Engineering and IT Systems for Business” (NEITB). This module is taught during the 2^nd semester (summer semester) of the 4^th year of engineering studies as part of the EISC specialization and is composed of two elements (submodules) of 30 hours each: (E1) “Networks Engineering for Business” and (E2) “IT Systems for Business”. The objective of the first element (E1) is to be able to use technical tools and methods for analysis, design, planning and optimization of networks to support Enterprise business and services. Similarly, the second element (E2) aims, through transversal projects, to help students to acquire and develop expertise in software and IT solutions to improve the overall quality of service of the company (user’s/customer’s satisfaction, productivity and/or technical support related to all the market issues). Several reasons explain the choice of the NEITB module to test the flipped classroom . First, conducting the experiment at this academic level is consistent with the recommendation of the researchers in the field. For example, the study of Strayer (2012), much appreciated by the systematic review of literature in Bissonnette & Gauthier (2012), recommends using the flipped classroom with high graduate students. Strayer (2012) highlights the importance of conducting such advanced training involving highly motivated students who are passionate about new ICT capable of fulfilling the company’s needs (or requirements). In fact, this module has been introduced in the specialization ESIC following the request of our socio-economic partners and after an evaluation undertaken in 2014 (a survey of students, a high rate of the End of study traineeship in E1/E2 areas, statics of the insertion of our graduates and analytics about the market need, etc.). In addition to the academic level, the nature of the module (particularly E2) is suitable in using ICT and collaboration paradigms in the context of a flipped classroom.

As a result, we decided to conduct our experiment in teaching the NEITB module as follows: a traditional learning of E1 (teacher lecture and in-class courses followed by homework and applications) and a flipped learning of E2. This is justified in part by the great pedagogical flexibility proposed in the official E2 program which allows open activities (up to 60% of E2 time could be taken in activities outside of the classroom). On the other hand, conducting the experiment in the context of the two close elements (E1 and E2) during the same period (summer semester, 2016), under the supervision of the same teacher and for the same class whose students have the same prerequisites, will give us the good opportunity in a term of comparison. It would be relevant when analyzing the results in order to put objectively into evidence the advantages and the limits of the flipped classroom as compared to the traditional learning in the field of engineering. In this way, we believe that our experiment is consistent with the criteria of Bissonnette & Gauthier (2012) and take into account the criticisms made towards the experiment of Pierce & Fox (2012), which used two reference groups (2011 and 2012), partially heterogeneous with respect to their prerequisites.

The two elements, E1 and E2, were consciously selected in order to attenuate as much as possible any implicit bias or influence on the actions of students and their outcomes that could be only due to an invalid choice of teaching elements. In the most basic sense, the participants should be engaged in E1/E2 training with almost the same degree of interests and technical abilities. In our specific case, a careful balance of both E1 and E2 elements have guided our choice. On the one hand, E1 and E2 are technically very close and related. The concepts, examples, homework and labs were given in relation to the same context of application (i.e. ICT Engineering for Business and Enterprise). Moreover, E1 and E2 are relatively at the same level of the technical difficulty and we made sure that all students gained, in prior semesters, the academic knowledge and study skills needed to succeed them. On the other hand, the teacher tried his best to motivate his students for both elements in order to avoid unbalanced engagement or disproportional contribution in any of the two elements that might be a source of irrelevant results or a biased judgment during comparison. The high interest for the ICT job and the goals of both elements were clearly explained in the beginning of the module. The teacher underlined also that E1 and E2 are both compulsory elements and they are equally considered in the validation of the module (Module mark = 50%* E1 mark + 50%* E2 mark). In addition, he applied analogical modalities during E1/E2 evaluation. Under such circumstances, there are grounds to apply a pair-wise comparison of the flipped and conventional training in a right way to get fair outcomes. And of course, the more we performe similar experiments within different groups the more rationalized results could emerge.

3.2. Conducting the experiment of the flipped classroom

Designing a pedagogical scenario for a flipped classroom entails adapting learning methods. It is important to rethink the structure, presentation and organization of courses (objectives, target audience, planning, resources, etc.). It is also necessary to adapt the place and time constraints by offering flexible and combined solutions (in-classroom and out-classroom training, face-to-face support and online support, etc.). Finally, it is important to reorganize the structure of trainings to ensure a thoughtful allocation of roles between all actors in harmony with the chosen modes. In our study, 39 students enrolled in the 4^th year of ESIC specialization, participated in the experiment between February and June 2016 in which all student and the teacher know each other so well and worked together in pervious ESIC modules. A personalized model of SSC that supports our flipped classroom scenario and takes into account all of the above elements was developed and established as part of the NEITB training. In the following subsections, we first present how the experiment took place in time, then we introduce the types of pedagogical activities we have implemented during the various phases of this experiment according to the same chronology (Figure 1).

Figure 1
The personalized SSC model as a support of our flipped classroom scenario.

3.2.1. Phase I (at school): orientation meeting and introduction

In any training, the initiation and preparation of the student is an essential phase. Therefore, such a phase represents, in the flipped training approach, a great added value in the orientation of the all participants. Thus, a joint preliminary session was held, at the beginning of this training, to introduce the students to the main topics and concepts, namely Software and IT engineering for Business. During this very short phase, the pedagogical objectives were clearly stated and the planning of the various steps was presented. Then, the teacher used a traditional pedagogical model (in-class PowerPoint slide presentations) to give an overview of the E2 filed. This short and preliminary session did not tend to emphasize memorization of concepts but rather to introduce briefly the students to the field and topics to be addressed, in addition to the planning of the next steps. We preferred to conduct this phase as a face-to-face class instead of using online video resources. After all, the use of online resources offers more flexibility during this part of the flipped learning, but it tends to reduce the social interaction between the participants (i.e. students and teacher) during a critical time that should maximize the student’s motivation and understanding.

Thus, a first step in school has been essential to preserve these interactions between students, and promote their acceptance of the following steps of this process. It has also allowed conducting face-to-face orientation discussions between the students and the teacher, followed by a broad assessment corrected in the classroom to encourage students to become more involved in training, as proposed in several researches in this framework, see for instance (Toto & Nguyen, 2009).

At the end of this phase, our students (i.e a population of 39 persons) have composed freely 12 teams (9 teams of 3 students and 3 teams of 4 students). Since the domain modeling implies the discovery of six families of Software and IT solutions for business (Business Intelligence (BI), Enterprise Resource Planning (ERP), Customer Relationship Management (CRM), IT Asset Management (GPI), Content Management System (CMS) and E-Learning training for staff (E-learning)) and by respect to the SSC model that recommends the exchange of ideas as a catalyst element for collective learning, the 12 teams worked as 6 pairs. Each pair, composed of two teams, had the same subject among the six subjects ((1) BI (2) ERP (3) CRM (4) GPI, (5) CMS and (6) E-learning) but worked on it separately during the Search and Structuring time. After that, each team should present the topic in front of the class and defend its own production during the confrontation phase, together with the opposing team. The assignment of subjects to all pairs (i.e. 12 teams) was conducted via a consensus between the students themselves in a completely free manner and without any intervention of the teacher.

3.2.2. Phase II (at home): search and structuring

This phase is about realizing in teams a general study and preparing a comparative market study of an IT solution for the company depending on the subject. For each product listed, the following elements must be identified: The operating principle, the requirements (needed hardware and software), the cost, the advantages and disadvantages, a fictive or a real use case. The expected outcome of each team is a collective work of synthesis in the form of an oral presentation, a report in MS Word format and a Lab of the best solution chosen by the team for illustration, with comment and all justifications. The objectives of this “at home” phase can be grouped into two distinct levels in line with the Accreditation Board for Engineering and Technology (2013) recommendations.

• o

  Objectives regarding the acquisition of knowledge and skills: Through the literature search and structuring of ideas, we are looking to complete the training of students during the flipped phase (at home). And above all, to encourage our students to start taking online self-training in ordre to be able to update their knowledge at any time and whenever needed in their professional curriculum;

• o

  Objectives regarding communication skills: using presentations and labs, we are looking to train students how to benefit from communication and ICT tools to present a topic and technically demonstrate its use to a professional audience.

3.2.3. Phase III: confrontation (presentations & lab)

1. a

   Step 1 (at school): presentation of work

At the end of the oral presentations on a same subject, the two involved and competing teams (a pair) confront in front of the audience to answer questions and to debate controversial questions in order to justify their choices regarding the subject in question. At the end, the relevance of the presentation and answers of each team are subject to an immediate evaluation by all students and the teacher. It's important to note that to mobilize all the students until the last minute, the order of presentations was determined only at the beginning of the session of the presentation by the drawing of lots. Such team-to-team confrontation shall respect the principle of fairness to keep students engaged as long as possible.

1. b

   Step 2 (at home): select a product name on behalf of the two teams

During this stage, students have to suggest six practical workshops LAB-i where i ϵ (BI, ERP, CRM, GPI, CMS, E-learning). Students of the two competing teams (Team i (Ti) and Team i' (Ti')) must make the final choice of the most prominent software and IT products in the field of their subject to present in the form of a hands-on manipulation during a demonstration workshop LAB-i. The choice can be done either by consensus or by vote between Ti and Ti' during a one week period. This stage of Prelab aims to encourage students focusing on the same subject, who competed to defend their choice during oral presentations (step 1), to sit together to jointly carry out the LAB-i preparatory work, and at the same time, feel a sense of belonging in regards to this same subject. That kind of feeling, capable of strengthening the group's cohesion, is necessary for the good conduct of the practical workshops. The workshop to present on behalf of two teams Ti and Tj' must be finalized as a short document (e.g. a Worksheet) and submitted to the teacher during the next week for validation of its pedagogical relevance and technical feasibility during the scheduled time (a face-to-face session of 2H).

1. c

   STEP 3 (at school): practical demonstration workshops

To optimize the scheduling of the Labs, alongside the Labs of other modules, while respecting the selected composition, the teams were divided into 3 groups (G1, G2 and G3), with 13 students in each group. In the end, the LAB-i will be carried out successively in a randomly decided order during 6 sessions in the presence of the teacher (Figure 2). Each Lab-i will be led and supervised by a heterogeneous pair {Ti-Sj} ; {Ti’-Sj’} that consists of students (Sj and Sj’) representative from each of the two teams Ti and Ti' in order to strengthen collaboration, benefit from the expertise developed by both teams and to ensure, at the same time, that at least one of the supervisors worked previously in detail on the chosen IT product. This learning practice is founded on some paradigms of Peer instruction (PI) (Mazur, 1997) and used to enhance student understanding and setting up six workshops, from theories to practices.

Figure 2
The composition of LAB-i sessions’ tutors from both teams Ti and Ti' to supervise the workshops for 3 groups.

4. Results analysis and a first outcome

4.1. Methodology and assessment tool

4.1.1. Methodology of evaluation

In our research context, we tried to use a sufficiently thoughtful evaluation approach in order to validate in a real environment the hypotheses advanced by the state-of-the-art and our proposals concerning the flipped classroom. Our objective is to evaluate our approach through two successive and complementary assessments. The first mode is participatory in order to actively involve students and the teacher in a qualitative assessment based primarily on responses to questions that we proposed at the end of the experiment. The second mode is fully quantitative and is based on statistical analysis of the results obtained by students at the end of the NEITB module (E1 and E2).

In parallel to the research works conducted in recent years, we aim to provide an assessment of the flipped classroom founded on a rational collection and better-argued data from our experiments. Due to the limitations of space, all relevant issues cannot be fully discussed here. So this paper focuses mainly on essential aspects of our work as summarized in the following:

• ●

  Evaluation of the flipped classroom as an approach for ICT engineers training: It consists of measurement and analysis measure and analyze the relevance of the flipped training as experienced and perceived by the all actors (students and the teacher) during the experiment according to several criteria and assessment levels (motivation mode, the efficiency in the acquisition of knowledge, the comparison of traditional/flipped mode, time and effort, gender sensitivity, etc.);

• ●

  Evaluation of the SSC model as a support of the flipped classroom: The SSC model consists of three major phases that we have designed and implemented to customize the tasks of each stage to fit the flipped classroom with all its specificities in our case. Thus, our evaluation should also gauge how relevant the use of our SSC personalized model during the experiment and measure how well our individualized design carries an added value for the flipped classroom approach (collaborative and social interactions, the confrontation of the teams, the contributions of each member in the project, etc.);

• ●

  The measure of students’ performance in flipped engineers training: The goal here is to determine the progress achieved by students at the level of academic results of the NEITB module after using the flipped classroom. It follows that to concretely measure the impact of the flipped classroom on student performance, confirm that it is a success or failure factor, we just need to compare the scores of all students at the end of E2 compared to E1. Since the two learning situations (E1 and E2) are comparable and very similar, except for their learning mode, fluctuation of results can be attributed to the used mode. It is important to note that in practice, we can never get an absolute equivalence between the experimental situation and the baseline situation (indicator). Our experiment does not establish conclusively a relationship of “cause and effect” between the results and the impact of the flipped classroom in improvement of learning, but with all of similarity parameters that we have gathered during the two elements teaching, our approach tried to reduce as far as possible the uncertainty to confirm to some extent that any change (improvement or decline) in scores is almost due to the change of the learning method (i.e. Flipped learning).

4.1.2. Assessment tools

The experimentation of the flipped learning that we realized required tools to observe and collect data relating to the progress of the various phases, the assessment of the students, and performances during the flipped classroom compared to the traditional training. Multiple sources of information were used to analyze the results of this experiment, namely closed questions, open questions, the academic results of examinations (e.g. Final Exams) and recommended activities. To encourage students to respond spontaneously, all surveys were anonym. However, to analyze the results from the gender point of view, the students were asked to indicate their gender (male or female). Regarding closed questions, students had the choice to check specific answers, indicate the type of answers Yes / No or a number among the following scale: {1 (very poor), 2 (poor), 3 (average), 4 (good), 5 (very good)}. To avoid forced responses, the alternative “I don’t know” was always left as a possible choice for students. Two categories of questions were delivered: a category concerns the flipped classroom; a category concerns the SSC model and collaborative working during the flipped classroom; and other categories that are not the focus of this paper. The following section details the translation of the questions of both categories:

1. a

   Closed questions about the flipped classroom

   1. 1

      I find this flipped learning mode very clear and motivating?

   2. 2

      This flipped teaching mode helped me to work in self-learning with more autonomy and more interaction with the team?

   3. 3

      This teaching mode encourages me to engage in self-training projects and seminars without a teacher?

   4. 4

      Evaluate your current knowledge in relation to your topic (/ 5)?

   5. 5

      Evaluate your current knowledge regarding to other topics (/ 5)?

   6. 6

      How could be your knowledge if you were trained in a traditional way (/5)?

      • -

        Open questions about the flipped classroom?

   7. 7

      Propose a specific change that the teacher could do to help and improve learning for all students?

   8. 8

      Propose a specific change that the teacher could do to help and improve the learning of students in difficulty?

2. b

   Closed questions about the SSC model and collaborative work?

   1. 1

      The mode of SSH (Search, knowledge Structuration and confrontation with a competing team) seems interesting and encourages learning?

   2. 2

      The session of orientation has helped me to enrich my work?

   3. 3

      Did you like that both teams worked separately on the same project?

   4. 4

      Did you like that both teams worked together to realize the practical workshop (LAB) of the same project?

   5. 5

      Overall, how do you appreciate the effectiveness of your team in realizing your project?

   6. 6

      Overall, how do you appreciate the effectiveness of your team compared to the second competing team?

   7. 7

      How many of your team members participated actively in the realization of your project?

   8. 8

      How would you rate your contribution to this project?

   9. 9

      How would you rate the contributions of other members to this project?

   10. 10

       How would you rate the contribution of your partner (the member of the other team) to the practical workshop?

   11. 11

       Do you have a teammate in this project that was your teammate within another project (in pairs, in trinomial, etc)?

       • -

         Open questions about the SSC model?

   12. 12

       Give a specific example of something that your teammates have learned from you and could not learn without you?

   13. 13

       During the assessment, I rate some projects according to the personal relationship that I have with the students in my classroom?

   14. 14

       How much time have you spent on your project (e.g. hours per week)?

4.2. Results and discussion

In total, all students (i.e. 39 students composed almost of half females versus half males) have submitted their survey responses. The filling of responses was conducted anonymously, free and at the end of the last session of NEITB Module (June 2016). We considered that this moment is good for collecting the answers to avoid any influence of the attitudes and opinions of our students due to the conditions of the final exam or its results. The teacher also delivered a general appreciation and feedback of using flipped classroom. In the following sections, we present an overview of the results of the survey. To simplify and to improve the readability of the graphs and charts in this paper, the five point scaled measures of satisfaction were collapsed into three basic levels: Poor (i.e. Very poor/Poor), Average and Good (i.e. Good/Very Good).

4.2.1. Evaluation of the flipped classroom as an approach for engineers training

The results of Figure 3 highlights the very positive assessment of the majority of the students (over 92%) regarding the flipped classroom as a clear and motivating teaching mode that assists student to work freely with more autonomy and more team interaction. According to 95% of the population, it will encourage students to engage in self-training sessions and seminars without a teacher.

Figure 3
General assessment about the flipped classroom.

However, this positive assessment must be somewhat nuanced when comparing results from a gender perspective, where a minority of female students (around 10%) seems to reject this teaching mode. This indication forces us to reduce our ambitions to the lowest common denominator and to adopt a gradual shift to the flipped learning in the case of heterogeneous classes or mostly composed of female students. Open questions showed the same trend (80% of male students, but only 60% female students are in favour of the flipped classroom associated with the SSC model). In general, students confirm in their answers that the flipped classroom improve their skills, particularly in relation to the following: collaborative work, project management and distribution of team tasks, solving technical problems, production of content, communication, patience and punctuality.

Note that at this level about 30% of students have not given answers and the majority of female students that were against the approach claims the unfair distribution of tasks within their teams. Regarding the transfer and acquisition of knowledge, when both males and female students confirm their overall satisfaction to acquire knowledge of their own subject, it is remarkable to note that female students are this time more confident in what they know about other subjects (55% satisfied and 45% very satisfied) when half the males (57%) shows a dissatisfaction regarding their knowledge in relation to the other topics. In both cases, students confirm that general knowledge of other subjects is below the knowledge of their own subject, on which they did work. Even so, all students (males and females) are not in favour of a traditional mode of training to acquire the same knowledge, which once again confirms how interesting the flipped classroom started to be for these students after the first experiment (Figure 4).

Figure 4
The appreciation of students for knowledge’s transfer.

Furthermore, the majority of students confirm that they have worked approximately 2H-4H during 6 weeks in E2 training which is reasonable and meets the official program guidelines. In addition, the teacher appreciated very well the flipped mode compared to the traditional mode. According to him, the effort was significantly reduced during training of E2, especially in terms of preparation, update and and revision of slides. He appreciated also to be a training facilitator and a supervisor of active learning instead of giving lectures all the time.

4.2.2. Evaluation of the SSC model as a support of the flipped classroom

The integration of the SSC model in the flipped classroom was an opportunity to test the relevance of an atypical scenario compared to the baseline scenario (Betbeder et al., 2006). Moreover, unlike the basic form of the flipped classroom that starts generally at home with online video session then followed by some activities at the school, we started in our case with a face-to-face orientation followed by the 3-steps of the SSC model (i.e. search, structuring and confrontation).

This confrontation phase has alternated the work in-class (presentation and demonstration) and out-class (a proposal for a common workshop). We estimate that such alternation (in-class and out-class activities) was a source to better engage students and to develop an efficient collaborative and active learning. Thus, at the end of our experiment, we used the survey to get feedback on the model we have adopted. The survey result highlights that the majority of students appreciated the SSC model and the orientation session held face-to-face under the supervision of the teacher (Figure 5).

Figure 5
The general assessment of students about the model.

The same positive assessment was expressed by students regarding our proposal for a confrontation of opinions in a spirit of constructive collaboration (i.e. (1) a confrontation of opinions between the two teams followed by (2) a collaboration to jointly realized a practical demonstration workshop). But this time, male students have expressed a dislike of team-to-team collaboration in labs (Figure 5).

When we look in more detail at the different criteria of evaluation of the same model (e.g. team performances, performances of the opposing team, the contribution of the other members of the project, the binomial collaboration in practical workshop), we note that students rarely drop an appreciation below 80% (Figure 6).

Figure 6
The students’ assessment about the teamwork.

This result could either show a good overall satisfaction of various phases of the flipped classroom, as stated before or some difficulty in the evaluation of the peers in an objective way. Nevertheless, it is curious to note that a minority (16% of females and 10% of males) expresses a dissatisfaction of the peer performance during the workshop lab (the member of the opposing team). These results that came up about self-evaluation, group-evaluation, peer-evaluation and teamwork will be discussed in greater detail in the near future.

4.2.3. The measurement of student performance in flipped training

To increase the reliability of evaluation, other empirical data were used to better quantify and to compare the performance of students in both sub-modules (E2 and E1). The two elements had the same terms of examination (a homework and a written final exam) that took place in the same academic context during the summer semester (2016). After these assessments we obtained the following results (/20) of each student (Figure 7) where each axis shows the evolution of the student average score of E2 (the flipped classroom) compared to E1 (traditional learning). To comment the results in more detail from a gender perspective, performances of students are depicted in two categories (Males/Females) where (S-Fn and S-Mn) means respectively (n^th Female Student and n^th Male Student) (Figure 7). From the results, we note that except for the S-F15, S-M1, S-M4, S-M9, all students made positive progress and an improvement in terms of the score of E2 as compared to E1.

Figure 7
The student performance at the end of the E1 and E2 training (Females at the left, Males at the right).

Let ΔM, the so-called the index of performance, equal to the difference of the score between both training as follows: ΔM = ((E2 Score - E1 Score)/E1 Score). We use ΔM as a tangible indicator of the improvement of students’ performance. The result showed an overall increase and an improvement in average scores of about ΔM≈12.8%. Note that the performance of females (ΔM-Female≈15.11%) increased relatively compared to the performance of males (ΔM-Male ≈10.11%).

Although some students have indicated a dislike of flipped classrooms, the overall result showed a relative improvement due to the flipped learning (Figure 8). Scores of female students were translated from a density around 12/20 to a density around 14/20 with a quasi-distributed manner. The distribution of the male students’ scores displayed also a positive translation to a density around 13/20, but without achieving much progress as recorded in the female results. So, in 39 cases, the performance of students confirms that the flipped learning was beneficial to both genders, particularly to females’ students (Figure 8).

Figure 8
The distribution of student scores (Males and Females) at the end of E1 and E2 training.

To support our affirmations, a paired sample t-test was conducted to compare the score of learning in flipped and conventional conditions (the participants were the same in each condition (N=39)). The two following tables present the means resulting in pairs of observations (before and after the test), the standard deviation (SD) and the Standard Error Mean (SEM) we got using the IBM SPSS analytics tools.

From Table 1, we can see that there was a significant difference (Mean=1.54) in scores for Flipped Learning (M=13.45/20, SD=0.97) and without Flipped Learning (M=11.92/20, SD=1.05). We need to determine whether or not it is a significant difference. Once again, our p-value (referred in the Table 2 as Sig (2-Tailed)) or significance value is less than 0.05 (p~0.000), ensuring approximately more than 95% confidence in the results. In this case we would reject the null hypothesis assuming that the true mean difference between the paired samples is zero and conclude that there is a statistically significant difference between our two conditions (i.e. improvement of scores). This statistically significant difference confirms, under the conditions of our experiment specifically, that the flipped learning really does have a good effect on our students and helped to improve their performance. A two-way mixed ANOVA test was also conducted using SPSS tools to investigate the impact of gender and E1/E2 training on the student performance. Here also we found that there was a significant main effect of training, as stated before (F(1, 37)=64.43, P~0.000) and that in the overall, females scored higher on the E2 test than males. However, the interaction between gender and student performances failed to reach statistical significance in this test (F(1, 37) = 0.036, p =.851).

Table 1
Paired Samples Statistics of E1 and E2 conditions.

	Mean	N	Std. Deviation	Std. Error Mean
Score on Conventional Learning (E1)	11.9135	39	1.04785	.16779
Score on Flipped Learning (E2)	13.4487	39	.97067	.15543

Table 2
Paired Samples t-Test to analysis the statistical significance of improvement due to flipped learning.

	Paired Differences					t	degrees of freedom	Sig. (2-tailed)
	Mean	Std. Deviation	Std. Error Mean	95% Confidence Interval of the Difference
				Lower	Upper
Conventional Learning - Flipped Learning	-1.53526	1.17958	.18888	-1.91763	-1.15288	-8.128	38	.000

Besides, unlike the work of Faillet (2014), we did not observe any significant regression of the result of good students. All students have exceeded in the flipped classroom, the global average of traditional training (Figure 9, green circle) except for two students (S-F1 and S-M5), who had previously difficulties in their school education and in communication. On the other hand, the average calculated per team (the average score of its members) shows also significant improvements following the flipped classroom except for few teams at the left side of the spider (Figure 9) where these teams could not progress that much. So, there is a need for further research to find important factors of such light improvement (nature of the topic/project, the cohesion of the team, the level of cooperation, etc.).

Figure 9
Distribution of student scores (Females and Males) at the end of E1 and E2 training.

Finally, in regard to the team performance, it is important to indicate that, as noted from the survey, 80% of females and 58% of males worked together in the flipped classroom project with members with whom they have already worked in other projects.

After all, from all these results, our first experiment reflects the positive impact of the flipped classroom to enhance engineering education in the case of ICT. It gives us also strong indicators of the relevance of our approach that we have developed, including the SSC model that we have adopted.

5. Conclusion

In this paper, we have presented a personalized model (the so-called SSC model) of the flipped classroom for an ICT engineering training taught during the 4th year of engineering studies. The main objective of this work is to assess the efficiency of the flipped learning model as compared to the traditional lecture-based model. To this end, we conducted an experiment in the context of two close elements in the field of ICT engineering; one taught according to the traditional mode and the other with the flipped mode. Both elements are given during the same semester, under the supervision of the same teacher and for the same class whose students have the same prerequisites.

The proposed approach was evaluated through two successive and complementary assessments. The first one is a qualitative assessment based on responses to a survey distributed to the students at the end of the experiment. The second one is fully quantitative. It is based on statistical analysis of the results obtained by students at the end of their training. The evaluation results have shown that the majority of students appreciated the flipped classroom as an approach for ICT engineers training and the SSC model with the in-class introductory session. Moreover, our experiment confirmed the utility of the flipped learning on engineering education and also proved the positive impact of this teaching mode on students’ achievement. We obtained an overall increase of about 12.8% as a contribution of the flipped classroom for all the students compared to the reference in-class training, with a more pronounced improvement in the scores of female students compared to the male students. Future research can throw light on gender issues, the different aspects of the evaluation of the teamwork, self/peer assessment, and PI in the context of the flipped classroom. We hope to confirm in this way the extent to which the generalization of our approach to other areas of engineering education could be possible.

• How to cite this article: Aqqal, A., Elhannani, A., Haidine, A., & Dahbi, A. (2017). Improving the Teaching of ICT Engineering using Flipped Learning: a personalized model and a case study. Production, 27(spe), e20162274. http://dx.doi.org/10.1590/0103-6513.227416

References

• Accreditation Board for Engineering and Technology – ABET. (2013). Criteria for accrediting engineering programs Baltimore: ABET. Retrieved in 1 March 2017, from http://www.heri.ucla.edu/PDFs/accreditation/2013/ABET.pdf
  » http://www.heri.ucla.edu/PDFs/accreditation/2013/ABET.pdf
• Benayed, M., & Verreman, A. (2006). Evaluation d’un dispositif d’apprentissage collaboratif a distance (Paper 393). France: Université du Sud Toulon Var. Retrieved in 1 March 2017, from http://isdm.univ-tln.fr/PDF/isdm25/BenayedVerreman_TICE2006.pdf
  » http://isdm.univ-tln.fr/PDF/isdm25/BenayedVerreman_TICE2006.pdf
• Betbeder, M.-L., Cottier, P., Schmidt, C., & Tchounikine, P. (2006). Dialogue in context, towards a referential approach in collective learning. AI & Society: The Journal of Human-Centred Systems and Machine Intelligence, 20(3), 314-330.
• Bishop, J., & Verleger, M. A. (2013). The flipped classroom: a survey of the research. In Proceedings of the 2013 ASEE Annual Conference & Exposition, Atlanta, Georgia. Retrieved in 1 March 2017, from https://peer.asee.org/22585
  » https://peer.asee.org/22585
• Bissonnette, S., & Gauthier, C. (2012). Faire classe à l’endroit ou à l’envers (Traditional or fip teaching?). Formation et Profession, 20(1), 23-28. http://dx.doi.org/10.18162/fp.2012.173
  » http://dx.doi.org/10.18162/fp.2012.173
• Chiang, Y., & Wang, C. (2015). Effects of the in-flipped classroom on the learning environment of database engineering. International Journal of Engineering Education, 31(2), 454-460.
• Depover, C., Quintin, J. J., & De Lièvre, B. (2004). Analyse des effets de deux modalités de constitution des groupes dans un dispositif hybride de formation à distance. International Journal of Technologies in Higher Education, 1(1), 38-44. Retrieved in 1 March 2017, from http://ritpu.ca/IMG/pdf/ritpu0101_depover.pdf
  » http://ritpu.ca/IMG/pdf/ritpu0101_depover.pdf
• Diefes-Dux, H., Moore, T.J., & Imbrie, P.K., (2006). Assessment of team effectiveness during complex mathematical modeling tasks. In Proceedings of the 36th Annual Frontiers in Education Conference (pp. 1-6), San Diego, CA, USA.
• Dillenbourg, P., Baker, M., Blaye, A., & O’Malley, C. (1996). The evolution of research on collaborative learning. In E. Spada & P. Reiman (Eds.), Learning in humans and machine: towards an interdisciplinary learning science (pp. 189-211). Oxford: Elsevier. Retrieved in 1 March 2017, from www.sticef.org http://tecfa.unige.ch/tecfa/publicat/dil-papers-2/Dil.7.1.10.pdf
  » http://tecfa.unige.ch/tecfa/publicat/dil-papers-2/Dil.7.1.10.pdf
• Faillet, V. (2014). La pédagogie inversée: recherche sur la pratique de la classe inversée au lycée. Sticef Journal, 21, 651-665. Retrieved in 1 March 2017, from www.sticef.org
• Felder, R. M., & Brent, R. (2003). Designing and teaching courses to satisfy Engineering Criteria 2000. Journal of Engineering Education, 2(1), 7-25. http://dx.doi.org/10.1002/j.2168-9830.2003.tb00734.x
  » http://dx.doi.org/10.1002/j.2168-9830.2003.tb00734.x
• Lage, M. J., Platt, G. J., & Treglia, M. (2000). Inverting the classroom: a gateway to creating an inclusive learning environment. The Journal of Economic Education, 31(1), 30-43. http://dx.doi.org/10.1080/00220480009596759
  » http://dx.doi.org/10.1080/00220480009596759
• Light, P. H., & Mevarech, Z. R. (1992). Cooperative learning with computers: an introduction. Learning and Instruction, 2(3), 155-159. http://dx.doi.org/10.1016/0959-4752(92)90006-8
  » http://dx.doi.org/10.1016/0959-4752(92)90006-8
• Lucke, T., Dunn, P. K., & Christie, M. (2016). Activating learning in engineering education using ICT and the concept of ‘Flipping the classroom’. European Journal of Engineering Education, 0, 1-13. http://dx.doi.org/10.1080/03043797.2016.1201460
  » http://dx.doi.org/10.1080/03043797.2016.1201460
• Mazur, E. (1997). Peer instruction: a user's manual Upper Saddle River: Prentice Hall.
• Pierce, R., & Fox, J. (2012). Vodcasts and active-learning exercises in a ‘flipped classroom’ model of a renal pharmacotherapy module. American Journal of Pharmaceutical Education, 76(10), 196. PMid:23275661. http://dx.doi.org/10.5688/ajpe7610196
  » http://dx.doi.org/10.5688/ajpe7610196
• Roberge, A. (2016). L’apprentissage inversé: avancée ou régression? Retrieved in 1 March 2017, from http://cursus.edu/dossiers-articles/articles/18434/apprentissage-inverse-avancee regression/
  » http://cursus.edu/dossiers-articles/articles/18434/apprentissage-inverse-avancee
• Stoyanova, N., & Kommers, P. (2002). Concept mapping as medium of shared cognition in computer-supported collaborativer problem solving. Journal of Interactive Learning Research, 13(1-2), 111-133.
• Strayer, J. F. (2012). How learning in an inverted classroom influences cooperation, innovation and task orientation. Learning Environments Research, 15(2), 171-193. http://dx.doi.org/10.1007/s10984-012-9108-4
  » http://dx.doi.org/10.1007/s10984-012-9108-4
• Toto, R., & Nguyen, H. (2009). Flipping the work design in an industrial engineering course. In Proceedings of the 39th IEEE Frontiers in Education Conference (FIE) (pp. T4F-1-T4F-4), San Antonio, TX, USA.
• Van Boxtal, C., & Veerman, A. (2001). Diagram-mediated collaboration learning. Diagrams as tools to provoke and support elaboration and argumentation. In Proceedings of the European Perspectives on Computer-Supported Collaborative Learning (pp. 131-138), Maastricht, The Netherlands.
• Yildirim, T. P., Shuman, L., & Besterfield-Sacre, M. (2010). Model-eliciting activities: assessing engineering student problem solving and skill integration processes. International Journal of Engineering Education, 26(4), 831-845.

Publication Dates

• Publication in this collection
  2017

History

• Received
  01 Nov 2016
• Accepted
  06 June 2017