Print version ISSN 0104-9224On-line version ISSN 1980-6973

Soldag. insp. vol.22 no.1 São Paulo Jan./Mar. 2017

http://dx.doi.org/10.1590/0104-9224/si2201.03

Technical Papers

Welding Joint Features Extraction Algorithm for Laser Triangulation Sensors Applied to Root Pass Control

Algoritmo de Extração das Características da Junta para Sensores de Triangulação Laser Aplicados ao Controle do Passe de Raiz

1Universidade Federal de Santa Catarina – UFSC, Departamento de Engenharia Mecânica, Programa de Pós-graduação em Engenharia Mecânica – POSMEC, Florianópolis, SC, Brazil.

2Instituto Federal Catarinense, Unidade Tecnológica/Mecatrônica, Rio do Sul, SC, Brazil.

Abstract

Joint geometry measurement is a key task for welding automation. Among the sensors used for this purpose, the Laser Triangulation Sensors for welding (LTS) are the ones that best characterize the joint geometry. Root gap measurement is especially important for success in the automated root pass deposition, with production and quality benefits in different applications, such as pipeline construction for the Oil & Gas industry. The measurement of this feature is sensitive to noises generated by welding and joint surface reflexivity. The image processing algorithms for gap measurement available in literature do not have the desired characteristics of measurement error and flexibility for dimensional variability of welding joints. This paper presents a new algorithm for root gap measurement of “V” type joints using a linear adjustment technique. This developed algorithm was evaluated from measurement errors found in images captured before and during orbital welding procedure of a standard test piece. The same images were processed with pattern correlation algorithm and derivative algorithm, both presented in academic papers. The proposed algorithm presented the best results of measurement error, robustness and flexibility.

Key-words:  Adaptive welding; Control and welding automation; Intelligent systems; Orbital welding; Productivity

Resumo

Palavras-chave:  Soldagem adaptativa; Controle e automação da soldagem; Sistemas inteligentes; Soldagem orbital; Produtividade

1. Introduction

Although the advantages of welding processes automation are notorious, for some of the more delicate operations, which present conditions of greater sensitivity to instabilities imposed by the geometrical configuration of the parts and joint, such as variant joints, low accessibility, thin plates and root passes, replacing the skill and cognitive ability of the human welder is still a great challenge. The welder observes the joint condition and the behavior of the melt pool and changes the torch movement pattern and power source parameters. In root weldings case, gap observation is especially important for welding success. The root pass is the first pass of the union of two pieces prepared with a “V” bevel and the gap is the minimum distance separating the two pieces (Figure 1). Because it is the first and most time-consuming pass, it usually determines the progress of the production line or construction stages, one of the most affected being the construction of pipelines for transportation of Oil, Gas and By-products. The main challenge of the root pass is to perform the joining of the two pieces so that the gap is filled with a welding bead of acceptable penetration and geometry, in a regular manner throughout the weld.

In automated root welding procedures, the gap must be measured by a sensor [1]. The Laser Triangulation Sensor for welding (LTS) is one of the technologies that best characterizes joint geometry and gap during welding [2-4]. Unlike other systems, such as inductive or capacitive sensors [5] or mechanical probes [6], LTS allow joint geometry measurement. In principle, sensing systems using the arc itself as a sensor may also have the ability to read aspects of the joint geometry [7]. With laser sensors, however, it is possible to extract with greater precision and reliability the values of gap, chamfer angle and cross-sectional area, among others.

The LTS are composed of a measuring head, equipped with a camera and a laser emitter, among other important components shown below in the system description, and hardware and software for communication, control, data processing and image processing. In this last subsystem, the scope of this work is inserted, which objective is the development and evaluation of an algorithm to extract the joint characteristics in V type chamfers. The comparison of the developed algorithm (Linear Adjustment) with techniques described in the literature (Pattern Correlation [8]; Derivative Algorithms [9,10]) is also approached in this work.

1.1. LTS working principles

The LTS is composed of a laser emitter that projects a structured plane of light over the piece to be measured, resulting in the projection of a laser line on the joint. A camera equipped with a set of lenses and filter captures this line. Figure 1 shows the operating principle of the LTS and its main components. The image captured by the sensor is processed with different algorithms and digital filters to correlate the image of the laser line with the geometry of the chamfer. The sensor can be calibrated to transform the position of the laser line pixels in the plane of the image (u, v) into points of the measurement system in millimeters [11]. In automated welding procedures, the sensor is mounted in front of the torch to inform the system of the joint geometry before welding [12].

In addition to naturally present noise sources in the laser triangulation process for welding, optical systems measurements suffer from the noise generated by the electric arc luminosity, splashes and reflectivity of the pieces. The literature has consolidated some hardware solutions for the sensor to support welding interferences. These solutions make the sensor capable of supporting the aggressive environment, but do not eliminate the process noises altogether [4,13]. On the other hand, the part to be measured often has known characteristics that help processing, such as the chamfer type and its main characteristics [14]. The steps of this image processing in an LTS are organized in Figure 2.

Once captured, the image is segmented, filtered and the joint profile is extracted. The joint profile can be understood as the set of points of the image that best represents the laser line. In the step of extracting the joint characteristics, the joint profile is processed to measure its characteristic points. This information is used to calculate the geometric characteristics of the joint, such as the gap (Figure 2) [14, 15].

Academic papers present algorithm solutions for gap measurement that meet certain automation applications [8-10, 16-20]. These solutions have limitations that often result in increased gap measurement errors during welding or in the need for adjustments to the measurement settings for each new measured piece. The need for constant adjustments makes the process inflexible and may limit its application. In this work, a new image-processing algorithm for gap measurement called Linear Adjustment is presented. The results of the measurements with this algorithm are compared to gap reference values and results obtained with the algorithms of Second Derivative [10, 19, 20] and of Pattern Correlation [8]. The Second Derivative algorithm uses the variation characteristic of joint profile inclination to measure its geometry and find the gap value. The main characteristic of this algorithm is flexibility of application for V type chamfers. The Pattern Correlation algorithm uses preset patterns to be correlated with the joint filtered image and perform the measurements. Standards correlation is a technique used for measurement in joints with reflexive surface for its robustness to noises.

The Linear Adjustment algorithm uses different techniques to minimize noises in images captured during welding. Initially, the captured imaged is segmented with an adaptive threshold technique [10]. This technique consists in detecting the level of image background intensity to define the filter value. As an image I (u, v) of height H and width W, Equation 1 gives the background value B [10].

{B=1WM1+HM2(u=1Wv=1M1I(u,10v)+u=1M2v=1HI(u,10v))M1=Int(H10) , M2=int(W10) } (1)

Any spatter present in the image may show higher intensity than the background value found and remain in the image after the threshold is applied. The segmented image is then processed with a particulate filter to remove these spatters. The filter groups nearby pixels with similar characteristics by defining an object in the image. In this way, a group of objects generated by the images of spattering, reflections and the laser line can be defined [21]. The objects identified from spatter and reflection images are much smaller than the ones generated from the laser line image. This feature is used to identify and remove these noises. However, spatter that cross the laser line at the time of image capture, or reflections very close to the line, can be grouped to the line object and remain present after the threshold is applied.

The chamfer profile is extracted from the filtered image with the technique of multiple peaks and neighborhood criteria [13, 15]. In this technique, the luminous intensity profile of each column along the lines is analyzed to define the set of points that best represents the chamfer profile. The luminous intensity of each column varies according to the presence of the laser line or noise. Figure 3 shows the image of a filtered bevel with inverted colors for better viewing. The image highlights the luminous intensity profile in columns A, B and C. In column A, the luminous intensity profile contains only one peak. The position of this peak along the columns of the image represents the position of the laser line in pixel coordinate. Columns B and C have two luminous peaks, one of which is generated by the laser line and the other by a reflection present in the bevel or spatter crossing the image.

In images without noises, the peak of greater intensity correctly represents laser line position. However, in regions where there is presence of noise generated by reflections or spatter, the peak of higher intensity may wrongly represent line position. In these cases, the analysis of the different light intensity peaks present in a column can help define the point that best represents the laser line. According to Figure 3, this analysis cannot be performed based on peak value or peak format [22]. In this paper, the choice of the peak that best represents the laser line is based on previous knowledge of the chamfer type and the pixels neighboring the analyzed point. The most intense (Peak 1) and less intense (Peak 2) peak sets shown in Figure 4 are processed to identify the chamfer profile.

The joint profile is then processed so that joint characteristics are extracted. In this algorithm, joint characteristics are extracted based on the adjustment of straight lines to the chamfer profile. Figure 5 shows the adjusted lines R1, R2, R3, R4, and characteristic chamfer points X1, X2, X3 and X4. The straight lines are found from the bisquare adjustment method, which presents best result in the fit of straight lines on noisy signals [23, 24]. This method returns the adjusted line and the error value displayed between the bevel points and the line. Points perfectly aligned will result in a straight line with zero error. To define R1, a line formed by the points of column 0 to column n + 10 is set, with n = 0 initially. If the displayed error is below a cut-off value, n is incremented and a new line is adjusted. This algorithm repeats until the error exceeds the cut-off value by defining the end of R1 line. The R2 line is found by fitting a line to the points of the columns n to k + 10 where k = n initially. The value of k is incremented until the error of the set line exceeds the cut-off value. The R3 and R4 lines are met with the same technique, starting from the other end of the chamfer profile. The points X1 and X4 are found from the intersection of the lines R1 with R2 and R3 with R4. The points X2 and X3 are defined from the end of the lines R2 and R3.

3. Experimental Testing

In order to evaluate the performance of the Linear Adjustment algorithm in gap measurement, images were captured with a development sensor before and during the welding process of a standard test piece. The test piece used was composed of SAE 1020 9.6 mm thick carbon steel plates machined with a 60° bevel and 300.0 mm length between the attachment points. The plates were assembled in such a way that the gap between them was approximately 2.0 mm, with variations in this value due to the machining and assembly of the test piece. The gap reference value was measured along the joint, with a 0.05 mm resolution caliper, every 30 mm of joint length.

Figure 6A presents the test bench with welding power source (1), robot equipped with LTS (2) and image acquisition and processing computer (3). Figure 6B highlights the cartesian displacement mechanism robot (1) [25], the developed LTS (2), the test piece used in the tests (3) and the direction of welding speed (Vs) . The LTS uses a 1.3 MP industrial camera with Ethernet communication, equipped with set of lens with focal length of 16 mm and a 650 nm band pass filter. The laser projector used outputs 30 mW power and wavelength equal to 650 nm. The resolution of this sensor is 0.05 and 0.09 px / mm (pixels per millimeter) along the length of the laser line and the measurement height. The maximum measurement rate is 10 acquisitions per second, totaling a cloud of 13,000 points in this range (300 mm test piece length measured). The LTS was mounted 50 mm in front of the welding torch and 60 mm over the plate surface (vertically). The welding process used was the Controlled Short Circuit, indicated for root pass welding by enabling better control of the molten pool. The process parameters are presented in Table 1 [26].

Table 1 Welding movement and power source parameters.

Power source parameters Welding movement parameters
Wire Feeding Speed (Va)
[m/s]
Medium Current
[A]
Medium Voltage
[V]
Travel Speed (Vs)
[mm/s]
Oscillation Width
[mm]
Oscillation Frequency
[Hz]
Side stopping time
[s]
3.2 120 16.5 2.5 1 2.5 0.3

Two groups of 2300 images of the test piece were captured, one composed of images captured before the welding and another during it. The two groups were captured under the same conditions of movement and sensor configuration. The images were processed with the three algorithms and analyzed for errors and measurement failures. A measurement failure is defined when the algorithm fails to measure a gap value or when this value is greater than the maximum value of 10 millimeters (five times the standard gap value expected).

The error values were obtained from the comparison between the reference value and the filtered gap for the same test piece position. The applied filter eliminates abrupt variations and smoothes the measured gap according to Figure 7. The measured Gap [n] is first compared to the previous filtered gap GapF [n-1], checking whether the measurement shows a permissible variation. If the variation is less than 0.3 mm, the gap value is processed with the first-order exponential smoothing filter with α equal to 0.3 and stored in GapF [n]. The α value being 0.3 represents a cut frequency of approximately 343 Hz.

The gap measurement error in a group of images was obtained from the differences between the filtered gap and the reference gap in the positions according to Equation 2. Ref is the gap reference value in position n, GapF the filtered gap, Uo the standard deviation of the error module and t the coefficient of the Student distribution to 95% certainty in a set of 11 samples.

Error=1nn=0nGapF[n]Ref[n] tu (2)

4. Results and Discussion

Figure 8 shows the gap measurement results with the Linear Adjustment algorithm, the filtered gap and the gap reference values along the joint for images captured before welding (A) and during welding (B). The gap measured values and reference values were processed to generate the percentage of failure and measurement error of the algorithm for this joint. The same images were processed using the Linear Adjustment, Pattern Correlation [6] and Second Derivative [19] algorithms to define the percentage of measurement failures and measurement errors presented by those algorithms. In Figure 8B it is observable that with the introduction of welding noises in the captured images there was an increase of spurious points due to measurement errors of the algorithm.

The percentage of failed measurements and the calculated error for the Linear Adjustment, Pattern Correlation and Second Derivative algorithms are shown in Figure 9A and Figure 9B. The Second Derivative algorithm presented the highest percentage of measurement failures. Processing the images captured during welding, this algorithm presented measurement failures in 23% of the images. This data shows lack of robustness of the algorithm in the gap measurement of noisy images. Even in images without the presence of spattering and sensible luminous variations, the algorithm presented the biggest measurement error. This result is directly related to the principle of extraction of the chamfer characteristics used by this algorithm. The bevels profile derivative operation eventually amplifies the profile noise, which often impedes the algorithm operation. The algorithm uses filters to soften the profile and reduce the noise intensity, but ends up distorting the bevel profile and inserting coarse measurement errors.

The algorithm of Pattern Correlation was less influenced by the process noises, with a low percentage of failure in both sets of images. Measurement failures occurred in 8% of the images captured during the welding process. The operating principle of the algorithm is based on previously saved patterns that are correlated with each captured image to find the bevel`s interest points positions. Due to this operating principle, the algorithm is robust because it does not have to extract the joint profile and subsequently identify the interest points. However, the algorithm is extremely dependent on the creation of standards for measurement success. In this way, the patterns must be recreated at the beginning of each procedure, which renders the algorithm unproductive. In very long joints, where the geometry of the joint profile varies significantly, the algorithm tends to present significant measurement failures.

The Linear Adjustment algorithm presented a smaller gap measurement error in images with process noise. This algorithm also presented the smallest percentage of measurement failures, indicating its robustness in measurements with and without the presence of welding noises. Because the operating principle does not rely on previously saved patterns, the algorithm has the flexibility required for application in automated systems. The algorithm can measure the joint position and characteristics, such as the gap, without the need to be reconfigured to each welded piece as it happens with Pattern Correlation algorithm. The Linear Adjustment used to identify the lines makes the definition of interest points more robust than the other processes. The algorithm adapts to bevels with different characteristics, allowing the application in joints with significant profile geometric variation. In the procedures without welding noises, the linear adjustment algorithm presented only 1% of measurement failures. These few measurement failures resulted in a data set with a larger standard deviation, which resulted in a higher maximum error in the measurement procedure for images captured before welding.

5. Conclusions

The Linear Adjustment algorithm proposed in this paper presents a solution for image processing of laser triangulation sensors applied in the root pass welding control. This algorithm is indicated for measurements with interference of process noise for its robustness. The algorithm’s operating principle uses the joint profile points straightening to perform the joint characteristics measurements and is therefore called Linear Adjustment. The Linear Adjustment algorithm was compared with literature proposed algorithms for “V” type joints gap measurement, presenting better results in relation to failures and measurement errors.

The Second Derivative algorithm, which uses the profile slope variation of the laser line to perform measurements, was sensitive to the process noises, presenting increased errors when applied to images acquired during welding. The Pattern Correlation algorithm presented robustness when applied to images with process noise, but it was not very flexible to chamfer variations. This algorithm is extremely dependent on the correlation patterns used to identify and measure chamfer characteristics. In this way, it is indicated that the correlation standards are calibrated to each new test piece. The Pattern Correlation strategy should be applied in short length joints or with little dimensional variation of its profile.

Acknowledgements

To the Federal Institute of Santa Catarina for the incentives granted to Dr. Eduardo Bidese Puhl, to Petrobras for the financial support, to the entire team of LABSOLDA/LABMETRO - UFSC for the collaboration.

References

1 Liu YK, Zhang WJ, Zhang YM. A tutorial on learning human welder’s behavior: sensing, modeling, and control. Journal of Manufacturing Processes. 2014;16(1):123-136. http://dx.doi.org/10.1016/j.jmapro.2013.09.004. [ Links ]

2 Curless B, Levoy M. Better optical triangulation through spacetime analysis. In: IEEE Computer Society. Proceedings of the IEEE International Conference on Computer Vision; 1995; Cambridge, MA. Washington: IEEE Computer Society; 1995. p. 987-994. [ Links ]

3 Tarn TJ, Chen SB, Fang G, editors. Robotic welding, intelligence and automation: RWIA’2010. Heidelberg: Springer Berlin Heidelberg, 2011. [ Links ]

4 Gu WP, Xiong ZY, Wan W. Autonomous seam acquisition and tracking system for multi-pass welding based on vision sensor. International Journal of Advanced Manufacturing Technology. 2013;69(1):451-460. http://dx.doi.org/10.1007/s00170-013-5034-6. [ Links ]

5 Dutra JC, Bonacosrso NG, Santos DE, Hemmer MH, Silva RHG. Automating a wheel manufacturing operation. Welding Journal. 2014;93:76-84, 2014. [ Links ]

6 Garašić I, Kožuh Z, Remenar M. Sensors and their classification in the fusion welding technology. Tehnički Vjesnik. 2015;22(4):1069-1074. http://dx.doi.org/10.17559/TV-20131127091427. [ Links ]

7 Mendonça FK. Evolução da técnica de seguimento de junta via sensoriamento do arco para operações de soldagem em posições forçadas [dissertação de mestrado]. Florianópolis: Programa de Pós-graduação em Engenharia Mecânica, Universidade Federal de Santa Catarina; 2013. [ Links ]

8 Luo H, Chen X. Laser visual sensing for seam tracking in robotic arc welding of titanium alloys. International Journal of Advanced Manufacturing Technology. 2005;26(9):1012-1017. http://dx.doi.org/10.1007/s00170-004-2062-2. [ Links ]

9 Prada DL, Puhl EB, Pinto TLFC, Silva RHG. Algoritmo para extração das características do chanfro aplicado em sensores de triangulação laser para soldagem. In: Associação Brasileira de Soldagem. Anais do 42° CONSOLDA - Congresso Nacional de Soldagem; 2016; Belo Horizonte, MG. São Paulo: ABS; 2016. p. 1-9. [ Links ]

10 Xu D, Jiang Z, Linkun W, Tan M. Features extraction for structured light image of welding seam with arc and splash disturbance. In: Institute of Electrical and Electronics Engineers. Proceedings of the ICARCV 2004 8th Control, Automation, Robotics and Vision Conference; 2004; Kunming, China. USA: IEEE; 2004. p. 1559-1563. (vol. 3). [ Links ]

11 Santolaria J, Pastor JJ, Brosed FJ, Aguilar JJ. A one-step intrinsic and extrinsic calibration method for laser line scanner operation in coordinate measuring machines. Measurement Science & Technology. 2009;20(4):045107. http://dx.doi.org/10.1088/0957-0233/20/4/045107. [ Links ]

12 Pires JN, Loureiro A, Bölmsjo G. Welding robots: technology, system issues and applications. London: Springer-Verlag; 2006. http://dx.doi.org/10.1007/1-84628-191-1. [ Links ]

13 Muhammad J, Altun H, Abo-Serie E. Welding seam profiling techniques based on active vision sensing for intelligent robotic welding. International Journal of Advanced Manufacturing Technology. 2017;88(1):127-145. http://dx.doi.org/10.1007/s00170-016-8707-0. [ Links ]

14 Li Y, Wang Q, Xu D, Yan Z, Tan M. Recent developments on welding image processing and features extraction. Measurement and Control. 2007;40(5):139-145. http://dx.doi.org/10.1177/002029400704000502. [ Links ]

15 Hang K, Pritschow G. Reducing distortions caused by the welding arc in a laser stripe sensor system for automated seam tracking. In: Institute of Electrical and Electronics Engineers. Proceedings of the IEEE International Symposium Industrial Electronics; 1999; Bled, Slovenia. USA: IEEE; 1999. p. 919-924. (vol. 2). http://dx.doi.org/10.1109/ISIE.1999.798737. [ Links ]

16 Huang W, Kovacevic R. A laser-based vision system for weld quality inspection. Sensors. 2011;11(1):506-521. PMid:22344308. http://dx.doi.org/10.3390/s110100506. [ Links ]

17 Jae SK, Son YT, Cho HS, Kon KI. A robust method for vision-based seam tracking in robotic arc welding. In: Institute of Electrical and Electronics Engineers. Proceedings of the 10th International Symposium on Intelligent Control; 1995; Monterey, CA, USA. USA: IEEE; 1995. p. 363-368. http://dx.doi.org/10.1109/ISIC.1995.525084. [ Links ]

18 Yang SM, Cho MH, Lee HY, Cho T-D. Weld line detection and process control for welding automation. Measurement Science & Technology. 2007;18(3):819-826. http://dx.doi.org/10.1088/0957-0233/18/3/034. [ Links ]

19 Li Y, Wang YF, Xu D, Tan M. On-line visual measurement and inspection of weld bead using structured light. In: Institute of Electrical and Electronics Engineers. Proceedings of the IEEE Instrumentation and Measurement Technology Conference; 2008. Victoria, BC, Canada. Canada: IEEE; 2008. p. 2038-2043. http://dx.doi.org/10.1109/IMTC.2008.4547383. [ Links ]

20 Li Y, Li YF, Wang QL, Xu D, Tan M. Measurement and defect detection of the weld bead based on online vision inspection. IEEE Transactions on Instrumentation and Measurement. 2010;59(7):1841-1849. http://dx.doi.org/10.1109/TIM.2009.2028222. [ Links ]

21 Demaagd K, Oliver A, Oostendorp N. Practical Computer Vision with SimpleCV: the simple way to make technology see. Sebastopol: O'Reilly Media, Inc.; 2012. [ Links ]

22 Haug K, Pritschow G. Robust laser-stripe sensor for automated weld-seam-tracking in the shipbuilding industry. In: Industrial Electronics Society. Proceedings of the 24th Annual Conference of the IEEE; 1998; Aachen, Germany. USA: IEEE; 1998. p. 1236-1241 (vol. 2). http://dx.doi.org/10.1109/IECON.1998.724281. [ Links ]

23 Fox J. Applied regression analysis, linear models, and related methods. Thousand Oaks: SAGE Publications; 1997. [ Links ]

24 Barros ALBP. Revisitando o problema de classificação de padrões na presença de outliers usando técnicas de regressão robusta [tese de doutorado]. Fortaleza: Universidade Federal do Ceará; 2013. [ Links ]

25 Carvalho RS. Robô CNC para a automação da soldagem MIG/MAG em posições e situações de extrema dificuldade [dissertação de mestrado]. Florianópolis: Programa de Pós-graduação em Engenharia Mecânica, Universidade Federal de Santa Catarina; 2009. [ Links ]

26 Silva RHG, Dutra JC. Controlled short-circuiting MIG/MAG welding (CCC) - Process analysis tools. Welding and Cutting. 2009;3:162-167. [ Links ]

Received: February 10, 2017; Accepted: April 26, 2017

This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License which permits unrestricted non-commercial use, distribution, and reproduction in any medium provided the original work is properly cited.