Lecture capture and learner engagement strategies for industrial engineering distance education: results of a pilot program

Paper aims: This study examines issues affecting the efficacy of online course delivery using lecture capture technology. This form of instruction presents particular challenges when classes are comprised of a mix of students who participate synchronously and asynchronously. Originality: The study examines a pilot program focused on improving the delivery of lecture capture courses, providing a framework for the implementation of similar programs at other institutions. Research method: Program feedback as to the quality of the learning experience is obtained from the instructor, as well as from both synchronous and asynchronous online learners. Main findings: The results provide insights into how students engage with the instructor and with one another in a lecture capture setting. Effective instructional practices are also identified. Implications for theory and practice: The results provide important insights that can enhance the delivery of courses utilizing lecture capture technology across a range of educational contexts.


Introduction
The development of effective means for the delivery of online course content continues to play an increasingly important role in engineering education.Lecture capture technology is one of the earliest examples of online teaching and learning, a practice that is still widely used today.Lecture capture is generally defined as the capturing of some or call elements of a live lecture in digital format (Newton et al., 2014).Lecture capture courses can offer students a flexible schedule and feeling of control over their learning (Euzent et al., 2011).However, challenges remain with respect to the ability to effectively engage students, particularly in courses that are offered jointly to students in a synchronous or asynchronous nature.
A review of the literature suggests much of the research in this area has focused on the use of lecture capture in three general empirical settings: (1) face-to-face class sessions, particularly for high-enrollment courses, where enrolled students are able to watch the videos afterward; (2) fully online courses where lecture content is recorded and provided asynchronously to online students, who watch the videos at their leisure; and (3) fully online courses where the lecture content is provided synchronously, with the lectures often being provided concurrently to face-to-face students.For example, studies have examined the use of lecture capture as a means for on-campus students to review previous material (Brooks et al., 2014), as well as to provide supplementary information as a part of blended courses or in preparation for upcoming exams (Marchand et al., 2014).
Research is more limited with respect to online courses that are delivered purely through lecture capture, particularly those that include a blended mix of face-to-face, synchronous, and asynchronous learners.This study aims to provide insights from the perspective of a university where online courses typically include a diverse mix of students from these three groups.In addition, context is also provided as to general challenges that arise when delivering classes through lecture capture technology.
Technology support for students and the instructor can be a challenge in online courses.Obstacles to consider when teaching through lecture capture include slide content preparation, equipment setup, and lecture setup and delivery (Davis et al., 2009).Proper support is needed for instructors to easily access the lecture capture technology; this is essential when the classroom does not have automated lecture capture.From a student perspective, there are also various challenges, which include students demonstrating the requisite amount of self-discipline necessary for coursework completion (Euzent et al., 2011).
Student engagement has been shown to be critical in order to facilitate effective learning (Bryson & Hand, 2007).Prior research has indicated that four key factors are correlated with student engagement, including skills, emotional connection, participation/interaction, and performance.Of the four factors, participation/interaction was found to relate to various internal and external indexes for students.One internal index that drove student engagement was students having a theory of learning, and set goals, such as grade performance (Handelsman et al., 2005).In consideration of these issues, instructors must consider methods beyond traditional raising of hands and answering questions to engage students in the classroom.This may include instilling a culture of learning, fun, and interaction (Handelsman et al., 2005).Another aspect to keep in mind for courses offered to distance students is the demographic and geographic location of the students enrolled.For example, undergraduates who attend a lecture live on campus may have a different motivation for being engaged, compared to a graduate student, working full time, who views the recorded lectures from an out of state location.
Lecture capture gives students an active control over their learning, which can be an advantage especially when students are off campus.They have the ability to view a recorded lecture at different speeds, as well as multiple times.If students find a concept particularly challenging, they can view a recording and focus on that area by replaying what is needed (Newton et al., 2014).This active control on learning, as well as other factors, indicates a positive correlation with student performance.Clear audio has been found by instructors to be one of the most important aspects of lecture capture.
As technology becomes an increasingly larger part of instruction, it is important to consider the impacts of these technologies on the effectiveness of instruction.Lecture capture presents the potential challenge of focusing too much on the quality of the recorded material, rather than the effectiveness of instructional delivery to the students (Newton et al., 2014).For example, online courses can be subject to extended lecture periods in lieu of more interactive discussions or time for students to ask questions.
It is also important to acknowledge that some students prefer online as opposed to in-person lectures (Owston et al., 2011).Prior research has also shown that the efficacy of instruction, in terms of academic performance, can vary between in-class and online sections.For example, Roberts (2015) found that students with a higher grade-point average (GPA) did not show a discernible difference in performance between in-class and online participation.However, lower GPA students generally performed more poorly when participating online, a finding that may be related to internal motivation or other qualitative differences.
In consideration of the aforementioned issues, this pilot program aims to develop best practices for lecture capture courses, with particular emphasis on classes that include a combination of in-person and online learners.The program is oriented around the production of high quality lecture recordings for distance learners, as well as the provision of high quality teaching for both the on-campus and off-campus students.

Objectives
The overarching goal of this study is to address challenges related to online course delivery through the development, implementation, and evaluation of a pilot program to improve the effectiveness of instruction in courses using lecture capture technology through the university's colleges of engineering and liberal arts and sciences.The program aims to examine students' levels of achievement and satisfaction in such courses, as well as to better understand how students engage with their learning with the overall goal of better informing course improvement.
This pilot program entails the development of a framework wherein support personnel work in conjunction with the instructor to address potential barriers of the course from the perspective of the online students.
The support personnel are employed through a joint organization supported by these two colleges at the university.During the course of the semester, these staff work closely with the instructor to identify current online delivery strengths and weaknesses, and potential course improvement techniques.
Based upon lessons learned during this pilot program, recommendations are presented as to how to improve the framework for potential rollout on a large-scale basis at the university as part of broader efforts to encourage the implementation of best practices and improve the quality of course delivery using lecture capture technologies.Emphases are on strategies that effectively engage a diverse range of students, who participate either face-to-face or as distance learners either synchronously or asynchronously.

Empirical setting and methods
The pilot program is conducted as part of a graduate level course in industrial engineering, titled "Engineering and Systems Management".The course provides an introduction to core concepts that are fundamental to contemporary engineering management.The content covers a diverse range of topics, which include: decision trees and associated probabilities; personal issues and challenges; working with management; client and the project team; personality types; and documents/forms that are useful for the engineering manager.The course includes a variety of case studies that each student examines in preparation for facilitated discussions that occur as a part of each lecture period.Students are also responsible for completing a semester-long course project in small teams.The course is comprised of two sections: (1) an on-campus section; and (2) an online/distance section.The online learners have the option to join live during the scheduled class time, which meets one day per week for three hours.All online learners have access to using a web conferencing technology to join live and to participate during the class discussion (synchronously).Alternately, online learners can watch a recording of the lecture capture at their convenience (asynchronously).
During the 15-week industrial engineering course, the on-campus learners are provided a traditional face-to-face classroom environment where they can interact with the instructor and their classmates, and receive immediate feedback.The synchronous online learners are provided with a web conferencing technology that allows them to connect to the classroom where the learners can ask questions, share experiences, or leave comments.The asynchronous learners, who are unable to join live, watch the lecture recordings at their own convenience.
Synchronous learners are able to use a chat box through the web conferencing technology.This allows them to type their comments and thoughts any time during the lecture, and they are then addressed by the instructor.This pilot program enhances ways in which communication is engaging the on campus and distance learners with the instructor.
Engagement is enhanced by allowing online learners to communicate directly with the instructor, meaning in most cases the instructor can allow online students to directly communicate via audio using the web conferencing with the entire class.No special software is required for the students' computers since the built-in microphone for most computers is sufficient.Online student audio is normally muted, though the instructor selectively engages these students during lecture and allows for two-way audio communication.Despite this interaction capability, the instructor must also be perceptive to potential online learner comments that arise in the chat box, particularly since some students may prefer to only participate in discussion via this option.Even in such instances, the instructor can directly address these students in attempts to engage in verbal discussions with the online learners to ensure their continued participation.
The university is subscribed to a widely utilized video management system.A capture device (box) is installed in each classroom that supports lecture capture recordings.Each such classroom is also equipped with an instructor microphone and overhead microphones to obtain audio from face-to-face students.In addition to recording capabilities, the system also provides analytic capabilities to examine patterns with which students watch the videos, raise questions, and participate in various other course activities.
The lecture video recordings are processed and uploaded to the video management system within a few hours of each face-to-face meeting session.This system integrated with the university's learning management system (LMS).During each classroom lecture session, a producer is located in the back corner of the room, where they are responsible for monitoring each lecture to ensure the audio and visual components are effectively controlled, as well as to identify any technical challenges that must be accommodated.These staff are available to assist the instructor with the classroom technology, assist any of the online students joining live, and are available for audio control changes.All instructors have a PC or laptop located in the front each classroom.With either setup, the system allows for the lecture capture platform to record the information from the instructor's computer, including annotated notes that are uploaded to an online file-sharing system that students are able to access via the LMS.Data for this case study were collected using documentary analysis by means of observable behavior.The instructor, instructional support specialist, and an undergraduate research assistant all participated in reviewing the live lecture recordings to provide feedback from various perspectives.In addition, the online students were provided anonymous surveys at the beginning and at the end of the semester to document their experiences in the course, highlighting strengths and weaknesses.Of the 55 online students enrolled in the course, 25 (45%) completed each survey.
While the surveys sought feedback from the online students, the instructor's feedback was obtained through a review of the video content and completion of a self-reflection with the instructional support specialist.The reflection focused on how the instructor interacted with the students, specifically how he: (a) made himself available to online students; (b) communicated expectations and deadlines; (c) assessed student progress; (d) provided feedback; and (e) held students accountable.These questions are similar to those used in a study that examined Chickering & Ehrmann's (1996), "Implementing the Seven Principles: Technology as lever," and their implications for adult learners in online courses (Grant & Thornton, 2007).

Results and discussion
On the first day of class, the professional and academic standards for all learners are stated.While online learners and on-campus learners commonly face different situations and environments, they each know what is expected of them and a clear outline of how to succeed in the course is provided.Students understand they can expect professional, timely feedback on their course efforts.This approach is supported by prior research that indicates strong instructor-to-learner visibility such as, prompt instructor comments/feedback through email and discussions, and recorded lecture videos lead to higher online learner success and a better connection with the course (Savery, 2005).
To facilitate class discussion, the instructor provides many real-world examples, sharing personal experiences and encouraging the learners to share their own experiences, as well.This approach is supported by prior research that suggests relevant, meaningful interaction with students improves student performance (Joksimović et al., 2015).The instructor's teaching style and the diverse course topics, which covered a range of technical and professional issues (e.g., communication, teamwork, organization, time management) required extensive student-faculty and student-student communication, addressing an important gap among engineering students as noted by Muniz Junior et al. (2017).
The instructor also made extensive attempts to personally engage students whether they were participating in a face-to-face or online format.This is critical because students may be prone to get bored, skip class, and pay less attention when participating online if they are not actively engaged by the instructor (Sweeten, 2015).This was found to be less of an issue in this pilot program, which may be reflective of the fact the course is taught at the graduate level, including a number of working professionals who are more serious about their participation as compared to early undergraduate students for example.
As a result of feedback from prior course offerings, the instructor of the subject course uses several different engagement strategies and methods to accommodate the learning environments for both on-campus and online learners.One critical issues that arises due to the mix of face-to-face, synchronous, and asynchronous students is effective communication within the peer group.The Engagement Theory proposed by Kearsley & Shneiderman (1998), proposes that technology can facilitate engagement in ways that are difficult to achieve otherwise.Within the pilot program, a group project is completed over the course of the semester and each group is comprised of an on-campus and off-campus student.These groups engage students of different age, background, and geographic location.The Engagement Theory suggests that students who will be working in group projects need a way to get to know each other quickly.This can be done through a short simple activity, or through the creation of short biographies and background statements.For example, have students get into pairs and discuss a problem or question (Kearsley & Shneiderman, 1998).
As a part of the subject course, the first day of class includes an exercise devoted to developing rapport within these groups where all students within the classroom are split up into pairs and exchange facts about one another.After some time, the students individually stand up and present the facts about their paired partner.Distance learners, who are watching live through web conferencing technology, also introduce themselves with a few short facts.This emphasizes communication and collaboration among peers and a community of learning, which have been shown to increase student motivation to learn.This gives students the opportunity to work with others of different backgrounds, diversities, and perspectives (Kearsley & Shneiderman, 1998).One limitation that was encountered here, and at other points throughout the course, was a difficulty in effectively engaging asynchronous participants.To address this concern and more effectively engage all students, one recommendation Production, 28, e20170078, 2018 | DOI: 10.1590/0103-6513.201700785/10 is to pair the students up in advance.This would allow asynchronous participants to introduce themselves or a classmate through a short video clip, which could be submitted through the LMS.
Engagement of even the synchronous students is also a documented concern of lecture capture courses and another way through which this course effectively leveraged communication best practice was through the use of the chat box for synchronous learners.This mechanism allowed for feedback to be addressed in near real-time by the instructor.Students who are more social in nature may prefer the option to engage synchronously, but it does not necessarily imply they will outperform asynchronous students.To address the participation of asynchronous students, discussion boards and other forms of communication are encouraged over the course of the semester.One essential advantage of using the text-based communication is that it allows time for reflection and critical thinking (Garrison et al., 1999).Fitze (2006) found the use of written electronic communication to provide learners with enhanced understanding of technical terminology.Online learning environments can promote social presence while creating meaningful interactions through the use of web conferencing technology for live chats or online discussion boards.When the use of these tools is promoted, significant learning is more likely to occur (Dixson, 2015).
While effective engagement involves students putting in time, energy, thought, effort, and feelings, it is also composed of behaviors, and communication with others.Because of heavy emphasis on communication with others, reading discussion posts, emails, and content may not be enough to fulfill that need (Dixson, 2015).The social nature of learning suggests that only when we communicate with others on what we are learning do we apply, and reflect on the material, furthering our understanding to a knowledgeable/skilled level (Dixson, 2015).To this end, the course also includes a mandatory group project, which provides an opportunity for learners to gain experience working together by mixing the on-campus students and the online students in each group.This organized approach forces communication for all students involved to prepare for the workforce and distance/off-site communication.
During the course of the semester, the instructor would adapt their behavior in response to pedagogical and technical issues identified when reviewing lecture capture videos.This documentary analysis resulted in several changes to several subtle behaviors by the instructor, which included increased eye contact with the camera when speaking to the online learners.This created an environment where online learners, both synchronous and asynchronous, felt included in class discussions.As a part of the review procedures, the instructor also became aware that some of the questions and comments shared among the on-campus learners were inaudible within the lecture video.Subsequently, the instructor would repeat any student questions and summarize all student comments for the benefit of online learners.These visual and audio connections helped to create a more personal and engaging environment for online learners.The results of these self-reflections were found to be one of the primary benefits of the pilot program.Based upon these experiences, a rubric is planned for utilization in subsequent offerings, which will allow for the proactive identification of means by which the efficacy of instruction can be improved in near real-time.
In addition to the documentary analysis, a post-course survey sought to investigate the means by which students communicated with one another, as well as with the instructor over the duration of the semester.
Students were asked to evaluate how effectively they were able to communicate using the announcements section and discussion board of the learning management system (LMS), as well as the web conferencing tools provided in the course.Results, shown in Figure 1, suggest the announcements section effectively facilitated communication between students, as well as between the students and instructor.
Students preferred the discussion board as compared to the web conferencing tools.This finding generally supports prior research that shows students prefer text-based versus audio communication (Oomen-Early et al., 2008).However, it is important to acknowledge that research has also shown that perceptions of social and teaching presence are higher when utilizing more active forms of online communication, such as asynchronous video posts or synchronous videoconferencing (Clark et al., 2015).This is an area where further feedback is necessary in order to understand the settings under which specific forms of communication are most effective in a lecture capture format.
As a part of the group project, the instructor also encouraged online students to pre-record their part of the presentation and insert the recording into the group PowerPoint presentation.This was found to reduce the potential for technical malfunctions to occur and assists in the flow of the presentation.On-campus group members can present live, and play the online members' recordings with an easy transition.
At the end of the semester, students provided feedback as to the overall course structure.The student evaluations also sought to examine the overall value of the lecture capture course format, the quality of the video and audio communication, and the ease of navigation of the LMS from the student perspective.Results of these survey questions, which utilized a five-point Likert scale, are shown in Figure 2.
Overall, students were highly receptive to the overall structure of the class (4.64 out of 5.0) and felt the video (4.28) and, particularly, the audio (4.44) quality of the presentations exceeded expectations.As a result of feedback from this pilot study, the instructor has adopted the practice of repeating on-campus student questions and comments in a summarized format.This approach ensures that online learners can clearly hear and understand in-class comments, which can be a challenge due to non-optimal classroom microphone functionality.This addresses a long-standing concern of lecture capture courses as noted by Newton et al. (2014).The instructor now places greater emphases on speaking volume and word enunciation after self-assessment of the lecture recordings.
Students also found the course site to be relatively easy to navigate using the LMS.The course syllabus (4.52), lecture notes (4.04), and lecture capture video (4.08) all scored highly in this regard.One noted limitation was the students' ability to effectively engage technical support, particularly when problems arose with respect to the lecture capture video (3.74).This is an area where the instructional support organization can focus on improvements for future offerings.
Lastly, additional feedback was sought from online students as to those lecture capture practices that were most effective from their perspective as learners.Details of this feedback is provided in Figure 3. Several audial and visual elements that have been cited as concerns as a part of prior lecture capture courses at the university were identified and focused on as a part of the pilot grant activities.
At the conclusion of the course, students were asked to rate which of these practices were effective in improving the manner in which they learned the course content.The most important elements largely related  to the manner in which the instruction engaged students, both audibly and visually through the lecture capture software.For example, students appreciated being able to view the same content as their on-campus counterparts, as well as the instructor's ability to repeat both questions and comments from face-to-face students.These efforts helped the online students to feel a greater sense of connection to their in-classroom counterparts and highlight an important aspect of effective online instruction.

Conclusions and recommendations
As the use of lecture capture increases in higher education, best practices continue to develop.With these advances, significant correlation has been observed between student performance and active control over learning (Newton et al., 2014).Online learners offer challenges that on-campus students may not; for example, determining to what degree they are engaged in the class.However, often online learners provide a rich and diverse group with significant real-world experience that contributes substantially to the class.Ensuring these valuable learners are appreciated and engaged in class is a major focus of this pilot program.
Overall, this program sought to develop a framework by which the efficacy of online instruction using lecture capture could be improved through the use of periodic feedback over the course of the semester.The results from this study support efforts to identify and modify teaching strategies to engage online students within this non-traditional learning environment that includes three modes of delivery (face-to-face, synchronous, and asynchronous).Important insights were provided as to the means by which students communicated with one another, as well as with the instructor.The instructor also gained important insights as to how to better enhance the effectiveness of communication to both face-to-face and online students over the course of the self-assessment.Ultimately, the results of this pilot serve as a basis for subsequent instructor-led initiatives to improve the effectiveness of online education at the university.
Initial feedback from online students indicate that efforts and strategies to improve online delivery methodologies are appreciated and effective.This study demonstrates the opportunities and challenges that exist when using lecture capture technologies as a primary mode of content delivery for online students, and that various student engagement strategies can improve the education experience for online students.
Based upon these results, improvements to the program are recommended as best practices for similar programs at other institutions.Subsequent efforts should focus on student engagement with a focus to improve student learning and enhance teaching strategies, with specific emphasis on online students and the differences in educational attainment between students completing lecture capture courses in a face-to-face, synchronous, and asynchronous online role.At the university that was the focus of this study, the proposed efforts would include the broader deployment of a mixed-method quantitative framework.This would include refined supplemental qualitative data collected through survey instruments such as those discussed in this paper.To this end, there are a variety of evaluative metrics that can be used to assess the efficacy of student learning from the perspectives of the face-to-face, synchronous, and asynchronous students.Universities generally implement an end-of-semester course evaluation, which solicits feedback regarding the quality of various aspects of the course.This feedback would be broadly applicable to participants of all three types and would include topics such as communication/clarity, pace, organization, enthusiasm, comfort of learning environment, and ability to hold attention.Comparisons of feedback across learner groups (i.e., face-to-face, synchronous, and asynchronous) would be particularly useful for items such as how effectively the instructor is able to hold students' attention, the overall degree of interaction with the instructor and other students, attentiveness and interest in the course material, and ability to focus and pay attention.Differences in these metrics would be particularly informative in consideration of the online student's role (i.e., synchronous vs. asynchronous), which the university will assess by building upon a framework developed by Schreiner & Louis (2011).
In addition to students' perceptions of the quality of certain aspects of the course, lecture capture is well suited for analyses of the degree to which students engage with the online course materials (Brooks et al., 2014).For this pilot project, and for all lecture capture courses at the university, the video recording system is integrated with the LMS to allow for an examination of when, and for what duration, students engage in various online components of the course.A screenshot of the system is shown in Figure 4, which illustrates how the number of times students view various videos over the course of the semester.The instructor and support staff are able to track additional information, such as how frequently students are asking questions, how many times they are viewing each presentation, and when they are engaging with other online content, as well.
In Figure 4, there is a clearly pronounced peak in early October before the mid-term exam, followed by a precipitous drop in viewership immediately after the exam.The videos provide a clear benefit in that lecture capture allows for more effective preparation for exams, including allowing for clarification of points of confusion (Evans, 2008).Overall, one of the primary advantages of lecture capture as a delivery medium is that students are able to re-watch content that is difficult to understand (Traphagan et al., 2010).Based on the results from the pilot, it is recommended that these analytic tools are used for short-and long-term evaluations of how students interact with the technology.The use of these tracking capabilities will allow for answers to several questions of interest, including: • How often do students watch the videos and when do they tend to watch?
• How does video utilization vary among face-to-face, synchronous, and asynchronous students?
• How frequently do students participate in discussion periods?
• How frequent is student-faculty interaction, including asking questions during or after class?Interestingly, one noted disadvantage of lecture capture is the potential for students and/or instructors to feel more self-conscious when lectures are recorded (Elliott & Neal, 2016).As a consequence, they may be less likely to interact with the instructor.While this behavior was not found to be an explicit issue in the pilot course, the instructor consciously attempted to actively engage both the face-to-face and online students, creating a more welcoming environment.
Another concern with lecture capture delivery is the degree to which students interact with one another.This is primarily an issue for online students, particularly those who participate asynchronously.The literature has shown such students to feel disconnected from both the instructor and from fellow students (Larreamendy-Joerns & Leinhardt, 2006).As such, group exercises that encourage interaction (e.g., the introductory exercise at the beginning of the semester noted previously) between the face-to-face and online learners appears to be critical to effectively engaging the full spectrum of learners.This will help to establish an environment that encourages student engagement and higher order thinking (Bryson & Hand, 2007).
Additional insights may also be gained through qualitative data collected by means of focus group interviews.Such interviews could be conducted directly with face-to-face students and with the online students through video conferencing technology.Expansion of the pilot program on a broader scale will allow for an examination of how the qualitative data presented in this study related to more quantitative metrics (e.g., class performance on assignments, projects, exams) to assess ultimately impacts on student learning outcomes.The university currently implements a separate internal competitive program that provides support for converting a primarily face-to-face course into one which utilizes lecture capture.As such, future research is recommended to assess the degree to which learning is impacted by this conversion to lecture capture in consideration of the content area, the degree program (e.g., undergraduate vs. graduate), and the characteristics of the instructors and students.
Moving forward, there are a number of opportunities to investigate several of the issues identified as a part of this study in greater detail.Foremost among these is better understanding the challenges and opportunities afforded to synchronous versus asynchronous online students.While synchronous participation allows for more effective one-on-one engagement with faculty and other students, asynchronous participation allows students the ability to learn at their own pace and take more time for reflection (Giesbers et al., 2014).The continuation of the pilot program detailed here will allow for in-depth longitudinal investigations to better understand both the qualitative and quantitative impacts of lecture capture on teaching and learning.

Figure 4 .
Figure 4. Tracking system for student engagement and participation.

ERRATUM
In the article Lecture capture and learner engagement strategies for industrial engineering distance education: results of a pilot program, DOI number: http://dx.doi.org/10.1590/0103-6513.20170078,published in journal Production, 28: e20170078, page 1: Where it reads: "Arlette Ruffer" It should read: "Arlette Rufer"