Accessibility / Report Error

Development of an Augmented Reality environment for the assembly of a precast wood-frame wall using the BIM model

Desenvolvimento de um ambiente de Realidade Aumentada para montagem de parede pré-fabricada em wood-frame a partir de modelo BIM

Abstract

This article presents the development of an Augmented Reality (AR) application to assist in the assembly of a precast wood-frame wall, based on the BIM model of the wall execution sequence. The research study used the Design Science Research approach and its aim was to develop an AR application named "montAR" (version 2.0). This application offers a tutorial that combines a wall model visualized in AR in actual scale, followed by an audio with step-by-step instructions of the assembly process. Its applicability was simulated in a laboratory with the participation of volunteers (architecture and engineering students). Two visualization gadgets were used and compared: smartphones and smart glasses. The potentialities and difficulties of the use of the AR system were assessed through a questionnaire answered by the participants and through direct observation and result analysis by the researchers. The results demonstrated the potential of using AR for precast wall assembly. From a technological innovation perspective, this study emphasizes the potential use of AR as a technology suitable for training and for construction quality control.

Keywords:
Augmented Reality; Wood-frame; Assembly; Building

Resumo

O presente artigo apresenta o desenvolvimento de um aplicativo de Realidade Aumentada (RA) para auxiliar a montagem de uma parede pré-fabricada em wood-frame a partir da modelagem BIM da sequencia de execução da parede. A pesquisa foi delineada através da abordagem da Design Science Research e teve como objetivo desenvolver o aplicativo"montAR" (versão 2.0). O aplicativo oferece um tutorial que combina a visualização em RA do modelo de uma parede em escala real com áudio explicativo dos passos de montagem. Sua aplicabilidade foi simulada em laboratório com a participação de voluntários (estudantes de arquitetura e engenharia). Foram utilizados e comparados dois dispositivos de visualização: smartphone e smart glasses. As potencialidades e dificuldades do uso do sistema de RA foram verificadas por meio de questionário junto aos voluntários participantes e pela observação direta e análise dos resultados pelos pesquisadores. Os resultados demonstraram o potencial do uso de RA para montagem de paredes pré-fabricadas. De uma perspectiva de inovação tecnológica, este estudo enfatiza o potencial de uso de RA como uma tecnologia para treinamento e para controle de qualidade da construção.

Palavras-chaves:
Realidade Aumentada; Wood-frame; Montagem; Construção Civil

Introduction

The light wood-frame construction method, or simply wood framing, combines a commitment to the environment with new construction techniques. Around 95% of the residences in the United States rely on the aforementioned technology, however in Brazil, its adoption is still incipient (MOLINA; CALIL JUNIOR, 2010MOLINA, J. C.; CALIL JUNIOR, C. Sistema Construtivo em "Wood Frame" Para Casas de Madeira. Semina: Ciências Exatas e Tecnológicas, v. 31, n. 2, p. 143-156, 2010.), restricted to some regions of the country. The embracement of new construction technologies, like wood framing, depends on individuals' training to develop and execute projects that involve such technologies. Hence, it is important to highlight that new skills must be gained and applied to the work.

For beginners, doubts and the inevitable occurrences of failures in the wood-frame panel assembly could be minimized with the aid of Augmented Reality (AR). As a tool to assist visualization, AR can contribute to the construction of wood-frame buildings as it exhibits additional information that enables better comprehension and also gradually guides the execution of the project. Embedded information into virtual models can be used if connected to the real construction elements and visualized in AR. This way, AR can provide a visualization of the assembly procedure in an interactive way at the same time it shows superimposed models onto the real world. By allowing individuals to be aided in the assembly process of wood-frame edifications, the AR technology would act as a facilitator to the diffusion of this construction system.

As an initiative to teach people about this construction method in Brazil, the application "montAR" (version 2.0) was developed. It offers a tutorial to aid the assembly of a wood-frame wall panel. This tutorial comprises a wall modeled in a Building Information Modeling (BIM) software with a real scale visualization through AR, accompanied by audio step-by-step assembly process instructions. The application was developed for two different mobile devices: smartphone and smart glasses with Android operational system. Smart glasses are a type of optical Head Mounted Displays (HMD) that integrates first-person cameras with hands-free displays (ALVAREZ CASADO et al., 2015ALVAREZ CASADO, C. et al. Face Detection and Recognition For Smart Glasses. In: INTERNATIONAL SYMPOSIUM ON CONSUMER ELECTRONICS, Madri, 2015. Proceedings... Madri: 2015.). As in the smartphones, the smart glasses already have the processing power and an integrated operational system together with tracking technology (usually GPS and 9 axis head tracking), battery, memory, storage, audio output, image and video capture and display.

The aim of this paper is to evaluate the potential of the use of AR for precast wall assembly and identify difficulties and limitations associated with the assessed gadgets: smartphone and smart glasses. The next section presents some researches involving assembly tasks supported by AR technologies.

AR as a tool for assembly tasks

AR is a technology that combines virtual elements with the real world in real time. The Augmented Assembly is the application of AR in an augmented environment where virtual objects are combined with the real world to enhance the artifact assembly (NEE et al., 2012NEE, A. Y. C. et al. Augmented Reality Applications in Design and Manufacturing. CIRP Annals-Manufacturing Technology, v. 61, n. 2, p. 657-679, 2012.). In the assembly processes, two or more objects are joined. AR use is being explored to guide the tasks required to put together parts of an artifact. In an augmented environment, physical parts of an artifact, real feedback and virtual contents are used to aid assembly of planned artifacts, allowing the benefits of both physical and virtual worlds.

In the last five years, numerous research studies have been conducted on the use of AR to assist assembly tasks. AR has been applied to, e.g., an animated system to improve the construction assembly process (HOU et al., 2013HOU, L.; WANG, X. A Study on the Benefits of Augmented Reality in Retaining Working Memory in Assembly Tasks: a focus on differences in gender. Automation in Construction, v. 32, p. 38-45, jul. 2013.); an experiment focusing on verifying the performance in retaining work memory according to the user gender (HOU; WANG, 2013WANG, Z. B.; ONG, S. K.; NEE, A. Y. C. Augmented Reality Aided Interactive Manual Assembly Design. International Journal of Advanced Manufacturing Technology, v. 69, n. 5-8, p. 1311-1321, 2013.); assisted interactive manual assembly design systems (WANG et al., 2013WANG, Z. B. et al. Assembly Planning and Evaluation in an Augmented Reality Environment. International Journal of Production Research, v. 51, n. 23-24, p. 7388-7404, 2013., WANG; ONG; NEE, 2013WANG, Z. B.; ONG, S. K.; NEE, A. Y. C. Augmented Reality Aided Interactive Manual Assembly Design. International Journal of Advanced Manufacturing Technology, v. 69, n. 5-8, p. 1311-1321, 2013., ONG; WANG, 2011ONG, S. K.; WANG, Z. B. Augmented Assembly Technologies Based on 3D Bare-Hand Interaction. CIRP Annals-Manufacturing Technology, v. 60, n. 1, p. 1-4, 2011.); a piping assembly in order to examine physical and cognitive potential of the assemblers (HOU; WANG; TRUIJENS, 2015HOU, L.; WANG, X.; TRUIJENS, M. Using Augmented Reality to Facilitate Piping Assembly: an Experiment-Based Evaluation. Journal of Computing in Civil Engineering, v. 29, n. 1, p. 5014007, 2015.); a building structure model assembly using wooden blocks in order to teach abstract construction and civil engineering topics in a more practical manner (SHIRAZI; BEHZADAN, 2015SHIRAZI, A.; BEHZADAN, A. H. Content Delivery Using Augmented Reality to Enhance Students' Performance in a Building Design and Assembly Project. Advances in Engineering Education, v. 4, n. 3, p. 1-24, 2015.). These researches are summarized in Figure 1. The localization technology aims to determine where to display digital content. Its accuracy possesses limitations on how AR applications can superimpose virtual information onto the real world (CHI; KANG; WANG, 2013CHI, H. L.; KANG, S. C.; WANG, X. Research Trends and Opportunities of Augmented Reality Applications in Architecture, Engineering, and Construction. Automation in Construction, v. 33, p. 116-122, 2013.). The hardware device can determine the level of mobility provided, so it is important to observe the display and processing device used. The control mechanism specifies the interaction techniques used to interact with the system, such as gestures or indirect controllers.

Figure 1
Recent AR researches involving assembly activities

Researches aiming to study assembly tasks normally adopt flat images as a marker in order to use optical tracking technologies. These markers are recognized in real-time and used as a reference object to precisely superimpose virtual 3D content onto the camera's live view. The fiducial markers are rectangular 2D markers with a fixed structure and distinctive black border that can be easily and very robustly detected and tracked (KOCH et al., 2014KOCH, C. et al. Natural Markers For Augmented Reality-Based Indoor Navigation and Facility Maintenance. Automation in Construction, v. 48, p. 18-30, dez. 2014.). The picture marker is an arbitrary borderless image (containing enough visual content) that can be robustly detected and tracked.

Some researches have explored assembly with bare-hand interactions using AR (WANG et al., 2013WANG, Z. B.; ONG, S. K.; NEE, A. Y. C. Augmented Reality Aided Interactive Manual Assembly Design. International Journal of Advanced Manufacturing Technology, v. 69, n. 5-8, p. 1311-1321, 2013., WANG; ONG; NEE, 2013WANG, Z. B.; ONG, S. K.; NEE, A. Y. C. Augmented Reality Aided Interactive Manual Assembly Design. International Journal of Advanced Manufacturing Technology, v. 69, n. 5-8, p. 1311-1321, 2013., ONG; WANG, 2011ONG, S. K.; WANG, Z. B. Augmented Assembly Technologies Based on 3D Bare-Hand Interaction. CIRP Annals-Manufacturing Technology, v. 60, n. 1, p. 1-4, 2011.). In these researches, the users manipulate and assemble virtual components, which are projected through a HMD. The virtual objects are manipulated by the movement of the thumb tip and index fingertip. The marker is used only to place the virtual content on the right position. Other researches use a mouse (HOU et al., 2013HOU, L.; WANG, X. A Study on the Benefits of Augmented Reality in Retaining Working Memory in Assembly Tasks: a focus on differences in gender. Automation in Construction, v. 32, p. 38-45, jul. 2013., HOU; WANG, 2013HOU, L.; WANG, X. A Study on the Benefits of Augmented Reality in Retaining Working Memory in Assembly Tasks: a focus on differences in gender. Automation in Construction, v. 32, p. 38-45, jul. 2013.; HOU; WANG; TRUIJENS, 2015HOU, L.; WANG, X.; TRUIJENS, M. Using Augmented Reality to Facilitate Piping Assembly: an Experiment-Based Evaluation. Journal of Computing in Civil Engineering, v. 29, n. 1, p. 5014007, 2015.) and touch screens (SHIRAZI; BEHZADAN, 2015SHIRAZI, A.; BEHZADAN, A. H. Content Delivery Using Augmented Reality to Enhance Students' Performance in a Building Design and Assembly Project. Advances in Engineering Education, v. 4, n. 3, p. 1-24, 2015.) as a control mechanism. Their studies targeted the comparison between paper-based manuals and AR systems as guidance to the assembly of: a LEGO robot (HOU et al., 2013HOU, L. et al. Using Animated Augmented Reality to Cognitively Guide Assembly. Journal of Computing in Civil Engineering, v. 27, n. 5, p. 439-451, 2013., HOU; WANG, 2013HOU, L.; WANG, X. A Study on the Benefits of Augmented Reality in Retaining Working Memory in Assembly Tasks: a focus on differences in gender. Automation in Construction, v. 32, p. 38-45, jul. 2013.); a piping system (HOU; WANG; TRUIJENS, 2015HOU, L.; WANG, X.; TRUIJENS, M. Using Augmented Reality to Facilitate Piping Assembly: an Experiment-Based Evaluation. Journal of Computing in Civil Engineering, v. 29, n. 1, p. 5014007, 2015.); and a scale model of a building structure (SHIRAZI; BEHZADAN, 2015SHIRAZI, A.; BEHZADAN, A. H. Content Delivery Using Augmented Reality to Enhance Students' Performance in a Building Design and Assembly Project. Advances in Engineering Education, v. 4, n. 3, p. 1-24, 2015.).

The use of AR assisting the assembly on civil construction activities is still limited. Except (HOU; WANG; TRUIJENS, 2015HOU, L.; WANG, X.; TRUIJENS, M. Using Augmented Reality to Facilitate Piping Assembly: an Experiment-Based Evaluation. Journal of Computing in Civil Engineering, v. 29, n. 1, p. 5014007, 2015.), all presented researches deal with scaled models, what makes it easier to manipulate. The research presented in (HOU; WANG; TRUIJENS, 2015HOU, L.; WANG, X.; TRUIJENS, M. Using Augmented Reality to Facilitate Piping Assembly: an Experiment-Based Evaluation. Journal of Computing in Civil Engineering, v. 29, n. 1, p. 5014007, 2015.) used real sized pipes to simulate a real instance. But even in this case, PVC pipes connected via insertion were used instead of welding iron pipes.

The AR implementation may not be suitable for all applications (NEE et al., 2012NEE, A. Y. C. et al. Augmented Reality Applications in Design and Manufacturing. CIRP Annals-Manufacturing Technology, v. 61, n. 2, p. 657-679, 2012.). Therefore, it is important to identify which applications would bring significant benefits to the user performance in terms of ease of learning new tasks, low rate of errors and decreased cognitive workload. In addition, it is important to analyze the effects of prolonged use of the devices and, from that, develop a highly interactive, efficient and customized interface for each application so that the user is satisfied, recognizes the advantages of the new technology and agrees to adopt it in the execution of their work (NEE et al., 2012NEE, A. Y. C. et al. Augmented Reality Applications in Design and Manufacturing. CIRP Annals-Manufacturing Technology, v. 61, n. 2, p. 657-679, 2012.). In the proposed use of AR, the users are likely to spend a substantial amount of time using the system; this is the reason the user experience must be studied and taken into account.

Materials and methods

The question leading this research is if and how AR could work as an assisting tool to the assembly of a precast wood-frame wall. The purpose is that an AR system could make the assembly stages and execution visible and explicit. Hence, the Design Science Research methodology was adopted in this research process.

The Design Science Research objective is to produce applicable and useful knowledge for problem solving, to enhance existing systems and to create new solutions or artifacts (LACERDA et al., 2013LACERDA, D. P. et al. Design Science Research: método de pesquisa para a engenharia de produção. Gestão & Produção, v. 20, n. 4, p. 741-761, 2013.). According to Peffers et al. (2007)PEFFERS, K. et al. A Design Science Research Methodology for Information Systems Research. Journal of Management Information Systems, v. 24, n. 3, p. 45-77, dez. 2007., this method allows the creation e evaluation of Information Technology artifacts designed based on a research problem. Piirainen and Gonzalez (2013)PIIRAINEN, K. A.; GONZALEZ, R. A. Seeking Constructive Synergy: design science and the constructive research approach. In: VOM BROCKE, J. et al. (Eds.). Design Science at the Intersection of Physical and Virtual Design. Berlin: Springer-Verlag, 2013. add that this type of methodology contributes to the existing knowledge by reaching for solutions to non-trivial questions in an innovative manner.

In short, the process of this research consists of five stages iteratively related:

  1. understanding wood-frame house construction process;

  2. objective: AR as a tutorial tool for construction tasks;

  3. design and development for AR usage;

  4. simulation; and

  5. evaluation.

The overall research process is represented in Figure 2. These stages carry relation to the Design Science Research stages suggested by Peffers et al. (2007)PEFFERS, K. et al. A Design Science Research Methodology for Information Systems Research. Journal of Management Information Systems, v. 24, n. 3, p. 45-77, dez. 2007., shown in the superior portion of the figure.

Figure 2
Figure The overall research process

The first stage of this research involved the study of related researches and also the understanding of wood-frame house construction process. A visit was carried out to a company specialized in the manufacture of wood-frame residences in an industrialized way in order to collect data to support the artifact's conception and development. Both the wood-frame panel assembly process in the factory and their installment on the building site were ascertained, Figure 3.

Figure 3
Working site of a wood-frame residence (left) and built-in hydraulic system (right)

This study sought to establish a parallel on how the AR technology could assist the panels assembly process allowing the self-construction. How could someone build wood-frame structures without specific technical knowledge about the subject? How could this construction system disseminate across Brazil? Could the AR function as a template for assembly?

The exploratory phase of this work allowed the researchers to familiarize themselves with the chosen construction technique and to comprehend the difficulties in the assembly process. This way, it was possible to establish a specific objective.

The development stage consisted of BIM modeling, assembly steps idealization, preparation of components and application development. This stage is briefly described in the following section.

After the development of the application, it was possible to simulate its use in laboratory and assess it with real users. Then, the applicability of the AR proposal was analyzed.

Design and development for AR usage

The aim of this section is to demonstrate the process of bringing the existing graphic information with a BIM model to mobile applications in order to offer a tutorial tool by means of AR.

BIM modeling and assembly steps

The aim of this research was to use BIM as a data source for an AR application, thus a BIM model was the primary data source to the development of the actual system. Therefore, a BIM model of a wood-frame wall was designed using the Autodesk® Revit® software. This stage was developed in two phases:

  1. wall structure; and

  2. built-in installations and wall sheathing.

In the first phase, the wall structure was designed. The proposed structure measures 1.70 meters wide by 1.75 meters high and possesses a central opening for a window. The structural framing material takeoff was extracted from Revit®, Figure 4.

Figure 4
Autodesk® Revit® print screen: wood-frame wall (left) and the structural framing material takeoff (right)

In the second phase, the built-in installations and wall sheathing were designed. The built-in installations were flexible conduit, electrical box, PVC pipe and elbow. Oriented Strand Board (OSB) was used as wall sheathing. The OSB panels provided were 1.22 x 2.20 meters. In the interest of exploring the most out of them, the cuttings were made after a study conducted on Revit®. From that, only 3 panels were necessary for both front and back sheathing. The complete material takeoff was extracted from Revit®, as presented in Figure 5.

Figure 5
Built-in installations and OSB wall sheathing (left) and the material takeoff (right)

The step-by-step was idealized through a BIM software by the "Phasing" resource. Hence, each step corresponded to a different phase in Revit®. In total, 10 phases were created, matching up with the 7 assembly steps and the 3 steps of inserting the pipe and conduit and placing the OSB panel.

Preparation of components for assembly

Molina and Calil Junior (2010)MOLINA, J. C.; CALIL JUNIOR, C. Sistema Construtivo em "Wood Frame" Para Casas de Madeira. Semina: Ciências Exatas e Tecnológicas, v. 31, n. 2, p. 143-156, 2010. point that, in Brazil, structural wood-frame projects use rapid growth reforestation wood as raw material, such as Pinus and, to a lesser extent, eucalyptus. According to Ferreira (2013)FERREIRA, R. MCMV de Madeira: conheça a tecnologia e os custos de construção do primeiro empreendimento em wood-frame do programa Minha Casa, Minha Vida. Construção, v. 146, p.16-21. set. 2013. Disponível em: <http://construcaomercado.pini.com.br/negocios-incorporacao-construcao/146/artigo299692-1.aspx>. Acesso em: 10 ago. 2015.
http://construcaomercado.pini.com.br/neg...
, the wood used is from reforestation and it doesn't go through any transformation process that requires energy, favoring the low consumption of energy in the manufacture of residences

In this research, Eucalyptus components, section 5x11 centimeters, were used. The eucalyptus was cut in accordance with the design. Each one was identified by a color according to its length and function.

Subsequently, the wall was physically assembled. For the first phase, the components were drilled in order to offer a pre-set assembly system. Letters were written alphabetically to identify elements of the same color to favor its location in the panel. For the second phase, the built-in installations and OSB wall sheathing were placed, Figure 6.

Figure 6
Built-in installations and OSB wall sheathing

Application development

Next, the artifact's functionality, characteristics and development are presented. The "montAR" application was developed in order to offer an assembly guide of a wood-frame wall. Using AR, the user's view is augmented with the assembly sequence, annotations and audio instructions designed to facilitate the task comprehension, positioning and execution.

Two versions of the "montAR" application were developed. The first consisted of a tutorial developed for Phase 1, associated with the wood-frame panel assembly. The 2.0 version adds Phase 2, which includes the built-in systems and OSB wall sheathing. The "montAR" application is available on Google Play1 1 Available at: <https://play.google.com/store/apps/details?id=br.com.marketingaumentado.montar&hl=pt_BR>. and on Moverio Apps Market2 2 Available at: <https://moverio.epson.com/jsp/pc/pc_application_detail.jsp?pack=br.com.marketingaumentado.montar&page=0&key=montar&cat=>. .

The "montAR" application was developed from the Metaio's Software Development Kit (SDK) for Unity 3D®. Its functionality is based on the use of an ID Marker (a kind of a fiducial marker) associated with the assembly execution steps of a wood-frame panel. When the ID Marker is seen in AR, it indicates the correct position of the pieces that make up the panel by displaying the model in real scale, highlighting its position on the assembly board. Markerless tracking techniques, such as tracking using the salient geometric features of real spatial objects, were considered to be adopted. However, as all of the wood components have the same rectangular shape, this approach was dismissed.

After the downloading, the "montAR" application could be accessed by its icon on the screen of the devices. When running, the application displayed a menu with the options: Phase 1, Phase 2 and Acknowledgement (in Portuguese). In Phase 1 and 2, there was a menu on the right side numbered from 1 to 7 to access the panel execution steps. These steps consist of the exhibition of the AR model elements and also of audio instructions that include the correct positioning, orientation (vertical/horizontal), joints and additional assembly referral, Figure 7.

Figure 7
"montAR" application: icon, initial menu and the steps to assemble the wall

It's important to notice that an assembly sequence is valid only if all the assembly operations are geometrically feasible (WANG et al., 2013WANG, Z. B.; ONG, S. K.; NEE, A. Y. C. Augmented Reality Aided Interactive Manual Assembly Design. International Journal of Advanced Manufacturing Technology, v. 69, n. 5-8, p. 1311-1321, 2013.), so the assembly sequence was established without interference between parts during the assembly process. In each step, through AR visualization, the wood pieces added are highlighted with its correspondent color over the assembly board, indicating, in real scale, the right position that the pieces need to be inserted. Hence, the BIM model had to go through a treatment process described by Cuperschmid and Ruschel (2013)CUPERSCHMID, A. R. M.; RUSCHEL, R. C. Desenvolvimento de Modelos 3D para Utilização no Aplicativo de Realidade Aumentada equipAR! In: SIMPÓSIO BRASILEIRO DE QUALIDADE DO PROJETO NO AMBIENTE CONSTRUÍDO, 3.; ENCONTRO DE TECNOLOGIA DE INFORMAÇÃO E COMUNICAÇÃO NA CONSTRUÇÃO, 6., 2013, Campinas. Anais... Porto Alegre: ANTAC, 2013., which includes: exportation file as FBX format, model optimization in the 3D Max® software, setup of the correct scale, materials application (colors, textures and transparency), alignment in relation to the marker position and ideal illumination of the virtual scene in Unity 3D®.

Additionally, audio instructions referring to each step were recorded. The Adobe Audition® software was used to bring optimization to the recordings. They were, then, exported as MP3 format into Unity 3D®.

In the Unity 3D®, the application interface was developed with an initial menu and a numbered menu with the assembly steps. The models and audio were associated with each step. Meshes were created corresponding to the measurements, so that the model, when seen in AR, could exhibit them, Figure 8. At the end of this process, the Unity 3D® project was compiled into an APK file (Android application package).

Figure 8
Model setup in Unity 3D® (left) and additional meshes (right)

The application was designed to run on Epson Moverio™ BT200 and on smartphones with Android operational systems, Figure 9. The Moverio™ BT200 is composed of binocular smart glasses (240g) that use two translucent screens to project the augmented content onto the surface of the real environment and a control pad (165g). It was used in this study because it: is capable of displaying three-dimensional models, provides a reasonable field of view (compared to other existing AR smart glasses at the beginning of the research) of 23 degrees, has a translucent optical display system, has a battery life of approximately 5.8 hours and was marketing available.

Figure 9
Smartphone (left) and Epson Moverio™ BT200 (right) use for visualization of the wood-frame wall

Through Metaio SDK, it was possible to encounter two ways of accessing AR resources in the Moverio™ BT200: (i) one in which the device captures the real scene by the glasses built-in camera, mixes the virtual content with the real scene and projects all of the content on the screen. In this situation, there is no possibility to use stereoscopy. (ii) and the other in which there is no capture of the real scene and the virtual content is included in the real scene. However, the insertion of a black background is needed, which causes partial occlusion of the real scene by the shading of the physical environment. In other words, the device always projects an image in 100% of the screen area, disallowing the resource of only adding the virtual content to the scene that is being visualized by a translucent screen. In this situation, there is the possibility to use stereoscopy. Given these possibilities, the first way of using the device was chosen in order to avoid the real scene occlusion, even though it offers a flat visualization.

Simulation and evaluation

The first version of the "montAR" application was used to evaluate the Phase 1, which was associated with the wood-frame wall panel assembly. Phase 1 was assessed using smart glasses (CUPERSCHMID; GRACHET; FABRICIO, 2015CUPERSCHMID, A. R. M.; GRACHET, M. FABRÍCIO, M. M. Realidade Aumentada como Auxílio à Montagem de Parede em Wood-Frame. PARC Pesquisa em Arquitetura e Construção, Campinas, v. 6, n. 4, p. 266-276, dez. 2015.) and smartphone (CUPERSCHMID; GRACHET; FABRICIO, 2016CUPERSCHMID, A. R. M.; GRACHET, M. FABRÍCIO, M. M. Augmented Reality as a Tutorial Tool for Construction Tasks: wood frame wall assembly supported by smartphones. In: INTERNATIONAL CONFERENCE ON ADVANCES IN COMPUTER-HUMAN INTERACTIONS, 9., Veneza, 2016. Proceedings... Veneza, 2016.). This paper proposes the comparison between the use of both devices for Phase 1 as much as for Phase 2, which includes the built-in systems and sheathing. For that reason, this paper contemplates the assessment of all the assembly process of a precast wood-frame wall, assisted by the "montAR" application (version 2.0) running on smartphone and smart glasses.

The evaluation was conducted in a carpentry lab equipped with Epson Moverio™ BT200 smart glasses, a Samsung Galaxy S5 mini smartphone (both with pre-installed "montAR" application), an ID marker, plus specific materials for both phases. During Phase 1 it was used: color identified eucalyptus studs, screws, an electric and a manual screwdriver and an assembly board on which the wood-frame wall panel should be mounted. During Phase 2 it was used: OSB, flexible conduit, electrical box, PVC pipe and elbow.

Formal research to evaluate AR interfaces involving users, such as this one, has not been widely performed (DÜNSER; BILLINGHURST, 2011DÜNSER, A.; BILLINGHURST, M. Evaluating Augmented Reality Systems. In: HANDBOOK of Augmented Reality. New York: Springer, 2011.). AR systems diverge from desktop systems in various aspects; the most essential is that such systems are produced for being used as a mediator or amplifier of human visualization (NILSSON, 2010NILSSON, S. Augmentation in the Wild: user centered development and evaluation of augmented reality applications. Thesis (Ph.D.) Linköping University, 2010.). One interface of AR includes the hardware (e.g. smartphone; smart glasses), the software (e.g. "montAR"), the devices for visualization (i.e. smartphone screen, smart glasses screen), the interface elements (e.g. menus, icons), the markers, the interaction technique (e.g. rotating the marker, pinching the virtual model), and the content shown in AR. Depending on the device, the tracking system, the interaction technique used and the interface in AR is altered. These aspects may justify, partially, the lack of well-established methods to evaluate AR interfaces.

To perform this evaluation, the method described by Olsson (2013)OLSSON, T. Concepts and Subjective Measures for Evaluating User Experience of Mobile Augmented Reality Services. In: HUMAN Factors in Augmented Reality Environments. New York: Springer, 2013. of subjective measures was adopted for evaluating the quality of the interaction with the system. The system's applicability was simulated in the laboratory with the participation of volunteers. The procedure to perform this evaluation was planned to happen as follows:

  1. presentation and Characterization: At first, the aim of the evaluation and the task goal (wood-frame wall assembly) was explained. Then, a characterization questionnaire of the participant was applied, checking the age, gender, education level, the frequency of smartphone or tablet usage and the previous use of any AR systems;

  2. appropriation of the technology by AR participants: The researcher demonstrated the functioning of the AR system. Next, participants used the application "montAR" for approximately 5 minutes, to familiarize themselves with this media language. During this period, the participants had the opportunity to handle the device and point it to the marker at the same time they visualized in AR the wood-frame wall model;

  3. wood-frame wall panel assembly: The proposal of this stage was to verify the quality of the participant's experience when using the AR system in a way to assembly the wood-frame wall. The users could freely walk around in order to observe the augmented environment from different perspectives and execute the task. The task was to assemble the wood-frame wall panel using the "montAR" application through Epson Moverio™ BT200 or Samsung Galaxy S5 mini. This task was planned to happen in two phases; and

  4. in addition to the questionnaire, the participant was asked to talk about the impressions, difficulties and opinions about the use of this technology. During the process, the researcher took notes and registered the process with photographs for further analysis.

Results and discussions

The assessment of Phase 1 took place in September 2015 and Phase 2 on February 2016. Two visualization gadgets were used and compared: smartphone and smart glasses. The evaluation conducted examines the quality of the user's interaction with the task. This evaluation aims a qualitative analysis of the interaction and manipulation of both the virtual information and the real wood studs for assembly. In this evaluation, subjective measures and qualitative analysis were performed. The quality of the user experience was evaluated through the categories described by Olsson (2013)OLSSON, T. Concepts and Subjective Measures for Evaluating User Experience of Mobile Augmented Reality Services. In: HUMAN Factors in Augmented Reality Environments. New York: Springer, 2013., an open question and direct observation.

The evaluation involved twenty-eight participants, among Architecture or Civil Engineering undergraduate students. Seven people at a time performed the evaluation of one phase with a distinct device (smartphone or smart glasses). The evaluation was conducted individually. All subjects voluntarily participated in the experiments. All of them had never used AR before. As AR training is equally effective for both males and females (HOU; WANG, 2013WANG, Z. B.; ONG, S. K.; NEE, A. Y. C. Augmented Reality Aided Interactive Manual Assembly Design. International Journal of Advanced Manufacturing Technology, v. 69, n. 5-8, p. 1311-1321, 2013.), in this study participated both genders, without the need of equal split.

All of the participants informed daily basis use of smartphone or tablet. None of them had previous experience in any kind of AR system neither had assembled a wood-frame panel before. Initially, the participants were given information about the research aim and explanation on the system operation. All of them took a few minutes to become familiar with the AR functioning before starting.

Results from the experiments indicate a positive effect of facilitations when using the AR system in assembly tasks. All participants demonstrated interest in the AR system used. The users could express their considerations about the application. For providing a realistic view of the users' motivation to actually use the application, both phases of the assembly process were very important.

Altogether, 28 evaluations were performed. All participants were able to assemble the wood-frame wall. In the first phase, each participant performed the assembly task in about 55 minutes and, in the second phase, 15 minutes. The first phase consisted of the structure assembly and the eucalyptus components used weighed around 10 kg each, which hampered the execution of the task. Also, the need to screw the elements together resulted in a longer completion time. In the second phase, less effort was required, as there were fewer steps and only the placement of the pieces was requested, without the need of nailing the OSB panels.

Users performing Phase 2 with smart glasses can be seen in Figure 10. It illustrates the effort to accomplish the task (from left to right): user standing in front of the panel to visualize the whole scene; user measuring the distance to place the PVC pipe; user placing the conduit; user positioning one part of the OSB; user holding the second part of the OSB with both hands; user putting the conduit onto the electrical box.

Figure 10
Smart glasses users performing the assembly of the wood-frame wall

Figure 11 exemplifies participants using smartphones in order to accomplish the task. The first three images demonstrate users visualizing the scene in the smartphone screen and the last three images illustrate users performing tasks with hands-free. In this situation, users visualized the scene through the smartphone screen, however, to have their hands free to perform the steps, it was necessary to memorize them. In this sense, the mental workload was higher than with the use of smart glasses.

Figure 11
Smartphone users perfoming the assembly of the wood-frame wall

The evaluations were categorized from two devices: smartphones and smart glasses. The results indicate the user's overall judgment of the experience. Despite the use of different devices and the addition of a new phase, the results of the questionnaire did not diverge much from ones from Phase 1 (CUPERSCHMID; GRACHET; FABRICIO, 2015CUPERSCHMID, A. R. M.; GRACHET, M. FABRÍCIO, M. M. Realidade Aumentada como Auxílio à Montagem de Parede em Wood-Frame. PARC Pesquisa em Arquitetura e Construção, Campinas, v. 6, n. 4, p. 266-276, dez. 2015., 2016CUPERSCHMID, A. R. M.; GRACHET, M. FABRÍCIO, M. M. Augmented Reality as a Tutorial Tool for Construction Tasks: wood frame wall assembly supported by smartphones. In: INTERNATIONAL CONFERENCE ON ADVANCES IN COMPUTER-HUMAN INTERACTIONS, 9., Veneza, 2016. Proceedings... Veneza, 2016.).

The results of the questionnaire, in general, indicate acceptance of the proposed system. All agreed that: the step-by-step provided was effective; the AR system was appropriate to the task; they felt satisfied with the way they performed and accomplished the task and they enjoyed the experience of assembling the wood-frame panel using the AR system. Regarding the screen size, most users declared that it was appropriate to the task. The interface provided by the AR system seemed natural to the majority of the participants. All smart glasses users and most smartphone users felt the desire to continue using the system. More than a half of the users felt involved in something extraordinary.

On the other hand, many did not feel comfortable when using the devices during the whole assembly period. The rejection was 50% for the use of smart glasses and 21% for smartphones.

Another factor that must be emphasized is the confusion caused by the superimposition of the virtual information onto the real world. About 21% of users of each device opted to abstain from opining.

In addition to the questions, an open question was provided destined to further considerations, where the participant could comment on his/her difficulties and suggest improvements on the system. Some important considerations, especially related to Phase 2, should be highlighted. The greatest objection involved the aforementioned visual confusion caused by the superimposition of the virtual information over the real one. A few number of users complained about the colors and textures chosen to the virtual components. Similar colors hampered the differentiation between real and virtual elements, Figure 12.

Figure 12
Similar colors and textures causing visual confusion

Some participants complained about the virtual image stability. At various moments, the virtual model would move in relation to the marker. All along the application development, the virtual model position was determined in order to avoid alterations, even if the user moved. Unfortunately, given the technological limitations of the hardware as much as software, this issue couldn't be totally bypassed, Figure 13.

Figure 13
Superimposition of the virtual model over the real one showing positioning flaws

Even though the Moverio™ possesses a frame for prescription lenses, they must be specially designed in order to fit the frame. For this reason, the participants who wore prescription glasses felt difficulty in using the smart glasses. They stated that using the smart glasses over the other was uncomfortable, Figure 14. In addition, the adaption of a strap holder to the smart glasses was necessary to keep it from falling off of the users' face, even those who didn't wear prescription glasses.

Figure 14
User wearing two glasses at the same time: smart glasses and regular glasses (left) and the strap holder attached to smart glasses (right)

Amongst the improvement suggestions, one user requested an animation showing the assembly process instead of using static models. Corroborating, other participant suggested the addition of visual orientations such as arrows indicating the position of the components. Other stated that not only the position of the pipe and conduit should be shown, but also information on how to insert them in the panel.

On the other hand, one complimented the system claiming that it is much better than paper-based manual. This participant stated that, with 2D manuals, one can visualize isolated parts when the AR enables the user to see the whole model. Other participant declared that the system is very interesting and didactic. The detailed visualization of each step was also complimented for facilitating the understanding of the assembly.

During the execution of the task of assembling the panel using AR, the participants' actions were registered. It was noted that the users would begin the task holding the smartphone and the Moverio's touch-pad. After a while, when the participants felt the need to have their hands free, they would place them in their pockets, Figure 15, and remove only when accessing a button to move to next step was necessary. It demonstrates an adaption process that involves both: the use of the technology and the task to be executed.

Figura 15
Initial use: holding the touch-pad (left) - after: touch-pad placed in the pocket and free hands (right)

The majority of the users preferred to use the headphone in only one ear, Figure 16. This attitude is probably related to the connection to the real world. Hence, for the sake of not losing the sound connection, many users received the audio information from just one ear. Several smartphone users preferred not to use the headphones, listening only through the device internal speaker.

Figure 16
Participant using only one headphone (left) and other listening through the smartphone internal speaker (right)

Conclusion and future work

In this article, we have presented the results of a user evaluation, in which a mobile AR system utilizing BIM models was created to assist the assembly of a precast wood-frame wall (including structure, built-in installations and OSB wall sheathing). This research is the first example of the use of AR for precast wood-frame wall assembly in actual scale.

An AR prototype system was developed based on a BIM model. The "montAR" application was successfully designed and used to conduct a simulation. Then, an evaluation devised to assess the user experience between the use of a smartphone and smart glasses was undertaken. The users had the opportunity to test a prototype of the system. This study provides a comparison between the mobile devices used in the AR technology for guiding people in a wood-frame wall panel assembly. The concrete experience of holding a smartphone or wearing smart glasses and using the application in real settings provided an important aspect of the user experience.

Although it has been observed that improvements can be made, the majority of users considered the application efficient as an assembly aid. None of the 28 participants had previous experience in assembling wood-frame structures, still all of them were capable of executing the task. The AR application enabled the understanding of the tasks and could be used as a tutorial tool for assembly.

The issues related to the difficulty in visualizing the model as a consequence of the adoption of certain colors, textures and transparency can be easily solved by altering them in the Unity 3D®. However, the greatest complaint resides in the discomfort of using the devices all along the process, especially the smart glasses. The precision and stability of the superimposition of the virtual model over the real one were also inadequate. Therefore, a combination of improved hardware and software should be achieved in the interest of using this technology in working sites.

Unlike Hou, Wang and Truijens (2015) that used cumbersome devices such as a PC, sensor, monitor and a large marker, this research required only smart glasses or a smartphone and a marker to work, which facilitates its use. This setup may favor its use in construction sites. Therefore, the knowledge produced, despite being used in the context of wall assembly in a laboratory, demonstrated potential application to the assembly of buildings in working site.

Furthermore, the study made enables the idealization of expanding this application to other fields, such as production control. Usually, methods of work verification are adopted, which include the use of checklists associated with in loco measurements to certify accordance with the initial project. In this respect, the exhibition of the virtual model superimposed onto the correspondent real elements in actual scale could reduce the inspection time and prevent errors. Therefore, in addition to the use as assembly guidance, the AR application could be used as a production control system, once it enables quick association between planning and execution. It is worth highlighting that the system offers assembly dimensions, which allows the checking of all elements positioning. As a tool for production control, a checklist associated with the virtual model could be added in order to ease the team communication and register important data, such as: who is responsible for verification, date of inspection, items approved and those that need repair, among others.

Future work should focus on adding support for the implementation of the AR system into real construction projects. The consolidation of AR in the assembly tasks in the construction sector still needs more tracking accuracy, virtual image stabilization, intuitive interaction techniques and suitable devices for construction sites (smart glasses more resistant with the capacity to project only the virtual image on the screen superimposed on the real world).

Acknowledgment

This research would not be possible without the co-operation of many people, and the authors would like to thank everyone who has participated in the evaluation section. We are very grateful for the financial support for the Junior Post-doctorate scholarship (Process: 151435/2014-6) and for the Scientific Initiation scholarship (Process: 149755/2015-5) provided by the National Council of Scientific and Technologic Development (CNPq).

References

  • ALVAREZ CASADO, C. et al Face Detection and Recognition For Smart Glasses. In: INTERNATIONAL SYMPOSIUM ON CONSUMER ELECTRONICS, Madri, 2015. Proceedings... Madri: 2015.
  • CHI, H. L.; KANG, S. C.; WANG, X. Research Trends and Opportunities of Augmented Reality Applications in Architecture, Engineering, and Construction. Automation in Construction, v. 33, p. 116-122, 2013.
  • CUPERSCHMID, A. R. M.; GRACHET, M. FABRÍCIO, M. M. Realidade Aumentada como Auxílio à Montagem de Parede em Wood-Frame. PARC Pesquisa em Arquitetura e Construção, Campinas, v. 6, n. 4, p. 266-276, dez. 2015.
  • CUPERSCHMID, A. R. M.; GRACHET, M. FABRÍCIO, M. M. Augmented Reality as a Tutorial Tool for Construction Tasks: wood frame wall assembly supported by smartphones. In: INTERNATIONAL CONFERENCE ON ADVANCES IN COMPUTER-HUMAN INTERACTIONS, 9., Veneza, 2016. Proceedings... Veneza, 2016.
  • CUPERSCHMID, A. R. M.; RUSCHEL, R. C. Desenvolvimento de Modelos 3D para Utilização no Aplicativo de Realidade Aumentada equipAR! In: SIMPÓSIO BRASILEIRO DE QUALIDADE DO PROJETO NO AMBIENTE CONSTRUÍDO, 3.; ENCONTRO DE TECNOLOGIA DE INFORMAÇÃO E COMUNICAÇÃO NA CONSTRUÇÃO, 6., 2013, Campinas. Anais... Porto Alegre: ANTAC, 2013.
  • DÜNSER, A.; BILLINGHURST, M. Evaluating Augmented Reality Systems. In: HANDBOOK of Augmented Reality New York: Springer, 2011.
  • FERREIRA, R. MCMV de Madeira: conheça a tecnologia e os custos de construção do primeiro empreendimento em wood-frame do programa Minha Casa, Minha Vida. Construção, v. 146, p.16-21. set. 2013. Disponível em: <http://construcaomercado.pini.com.br/negocios-incorporacao-construcao/146/artigo299692-1.aspx>. Acesso em: 10 ago. 2015.
    » http://construcaomercado.pini.com.br/negocios-incorporacao-construcao/146/artigo299692-1.aspx
  • HOU, L. et al Using Animated Augmented Reality to Cognitively Guide Assembly. Journal of Computing in Civil Engineering, v. 27, n. 5, p. 439-451, 2013.
  • HOU, L.; WANG, X. A Study on the Benefits of Augmented Reality in Retaining Working Memory in Assembly Tasks: a focus on differences in gender. Automation in Construction, v. 32, p. 38-45, jul. 2013.
  • HOU, L.; WANG, X.; TRUIJENS, M. Using Augmented Reality to Facilitate Piping Assembly: an Experiment-Based Evaluation. Journal of Computing in Civil Engineering, v. 29, n. 1, p. 5014007, 2015.
  • KOCH, C. et al Natural Markers For Augmented Reality-Based Indoor Navigation and Facility Maintenance. Automation in Construction, v. 48, p. 18-30, dez. 2014.
  • LACERDA, D. P. et al Design Science Research: método de pesquisa para a engenharia de produção. Gestão & Produção, v. 20, n. 4, p. 741-761, 2013.
  • MOLINA, J. C.; CALIL JUNIOR, C. Sistema Construtivo em "Wood Frame" Para Casas de Madeira. Semina: Ciências Exatas e Tecnológicas, v. 31, n. 2, p. 143-156, 2010.
  • NEE, A. Y. C. et al Augmented Reality Applications in Design and Manufacturing. CIRP Annals-Manufacturing Technology, v. 61, n. 2, p. 657-679, 2012.
  • NILSSON, S. Augmentation in the Wild: user centered development and evaluation of augmented reality applications. Thesis (Ph.D.) Linköping University, 2010.
  • OLSSON, T. Concepts and Subjective Measures for Evaluating User Experience of Mobile Augmented Reality Services. In: HUMAN Factors in Augmented Reality Environments New York: Springer, 2013.
  • ONG, S. K.; WANG, Z. B. Augmented Assembly Technologies Based on 3D Bare-Hand Interaction. CIRP Annals-Manufacturing Technology, v. 60, n. 1, p. 1-4, 2011.
  • PEFFERS, K. et al A Design Science Research Methodology for Information Systems Research. Journal of Management Information Systems, v. 24, n. 3, p. 45-77, dez. 2007.
  • PIIRAINEN, K. A.; GONZALEZ, R. A. Seeking Constructive Synergy: design science and the constructive research approach. In: VOM BROCKE, J. et al (Eds.). Design Science at the Intersection of Physical and Virtual Design Berlin: Springer-Verlag, 2013.
  • SHIRAZI, A.; BEHZADAN, A. H. Content Delivery Using Augmented Reality to Enhance Students' Performance in a Building Design and Assembly Project. Advances in Engineering Education, v. 4, n. 3, p. 1-24, 2015.
  • WANG, Z. B. et al Assembly Planning and Evaluation in an Augmented Reality Environment. International Journal of Production Research, v. 51, n. 23-24, p. 7388-7404, 2013.
  • WANG, Z. B.; ONG, S. K.; NEE, A. Y. C. Augmented Reality Aided Interactive Manual Assembly Design. International Journal of Advanced Manufacturing Technology, v. 69, n. 5-8, p. 1311-1321, 2013.

Publication Dates

  • Publication in this collection
    Oct-Dec 2016

History

  • Received
    14 Apr 2016
  • Accepted
    12 July 2016
Associação Nacional de Tecnologia do Ambiente Construído - ANTAC Av. Osvaldo Aranha, 93, 3º andar, 90035-190 Porto Alegre/RS Brasil, Tel.: (55 51) 3308-4084, Fax: (55 51) 3308-4054 - Porto Alegre - RS - Brazil
E-mail: ambienteconstruido@ufrgs.br