SciELO - Scientific Electronic Library Online

 
vol.18 issue1Body dissatisfaction among artistic gymnasticsadolescent athletes and non-athletesPhysical activity, BMI and metabolic risk in Portuguese adolescents author indexsubject indexarticles search
Home Pagealphabetic serial listing  

Services on Demand

Journal

Article

Indicators

Related links

Share


Revista Brasileira de Cineantropometria & Desempenho Humano

Print version ISSN 1415-8426On-line version ISSN 1980-0037

Rev. bras. cineantropom. desempenho hum. vol.18 no.1 Florianópolis Jan./Feb. 2016

http://dx.doi.org/10.5007/1980-0037.2016v18n1p93 

Original Article

Flexibility and muscle strength/resistance indicators and screening of low back pain in adolescents

Indicadores de flexibilidade e força/resistência muscular e a triagem de dor lombar em adolescentes

Rui Carlos Gomes Dorneles1 

Helter Luiz da Rosa Oliveira1 

Mauren Lúcia de Araújo Bergmann1 

Gabriel Gustavo Bergmann1 

1Federal University of Pampa. Uruguaiana, RS. Brazil

Abstract

Low back pain is increasing considerably among adolescents, reaching proportions similar to the adult population. Flexibility (Flex) and muscle strength/resistance, components of Health-Related Physical Fitness (HRPF), seem to be associated with presence of low back pain. The aim of this study was to assess the ability of Flex and muscle strength/resistance in the screening of adolescents with low back pain and to propose cutoff points for these HRPF components. The study was conducted with 1.455 students aged 10-17 years randomly selected. Low back pain was estimated using a questionnaire. Flex was measured by the sit and reach test and muscle strength/resistance by the one minute sit up test. The analysis of the receiver operating characteristic (ROC) curve was used to identify the capacity of Flex and muscle strength/resistance for screening low back pain. To propose cutoffs points, the best balance between sensitivity and specificity was analyzed. The frequency of low back pain was 16.1 %, higher (p<0.05) among females (21.6%) than in males (10.5%). The ROC curve analysis indicated that Flex is able to identify girls more likely of having low back pain (p<0.05). Sit and reach test and the proposed cutoff points can be used as screening tool of female adolescents more likely of having low back pain.

Key words Adolescents; Health; Low back pain; Roc curve

Resumo

A dor lombar vem aumentando consideravelmente entre os adolescentes, atingindo proporções semelhantes à da população adulta. A flexibilidade (Flex) e a força/resistência muscular (F/R), componentes da Aptidão Física Relacionada à Saúde (ApFRS), parecem estar associadas à presença de dor lombar. O objetivo do estudo foi verificar a capacidade da Flex e da F/R na triagem de adolescentes com dor lombar e propor pontos de corte para estes componentes da ApFRS. O estudo foi realizado com 1.455 escolares de 10 a 17 anos selecionados aleatoriamente. A dor lombar foi estimada através de questionário. A Flex foi medida através do teste de sentar e alcançar e a F/R pelo teste de abdominais em 1 minuto. A análise da curva Receiver Operating Characteristic (ROC) foi utilizada para a identificação da capacidade da Flex e da F/R para a triagem de dor lombar. Para a proposição dos pontos de corte, foi analisado o melhor ajustamento entre sensibilidade e especificidade. A frequência de dor lombar foi de 16,1%, sendo maior (p<0,05) entre o sexo feminino (21,6%) do que no masculino (10,5%). A análise da curva ROC indicou que a Flex é capaz de identificar meninas com probabilidade aumentada de apresentarem dor lombar (p<0,05). O teste de sentar e alcançar e os pontos de corte propostos podem ser utilizados como instrumento de triagem de adolescentes do sexo feminino com probabilidade aumentada de apresentarem dor lombar.

Palavras-chave Adolescente; Curva roc; Dor lombar; Saúde

INTRODUCTION

Health-Related Physical Fitness (HRPF) comprises cardiorespiratory fitness (ApC), muscle strength/resistance (F/R), flexibility (Flex) and body composition1. ApC and body composition seem to be more associated with cardiovascular health maintenance2,3, and F/R and Flex seem to be more associated with prevention of musculoskeletal health problems1,4. Thus, studies on HRPF levels of different population groups should be carried out.

A series of descriptive studies on HRPF of Brazilian children and adolescents have been developed indicating that a large proportion of these individuals does not meet HRPF recommendations5-7. The concern about low HRPF levels during childhood and adolescence is justified because its components remain relatively stable during the years of childhood and adolescence to adulthood8. In addition, regular practice of physical activity in this age group results in increased chance of maintaining these habits into adulthood, contributing to maintaining adequate HRPF levels9. The appearance of various risk factors for cardiovascular diseases is among the main health problems associated with low HRPF in children and adolescents3,10,11, strongly related to ApC and body composition, complaints of back pain and identification of postural deviations12 related to Flex and F/R. The presence of low back pain appears to influence the population’s quality of life and may even limit the performance of common tasks of the daily living13. In this context, low back pain in childhood and adolescence has attracted attention due to its high prevalence, with values similar to those found in adults14, being therefore a public health problem.

Although theoretical hypotheses support the importance of F/R and Flex for the prevention of low back pain in children and adolescents1,4, there are few studies that performed these associations and the results are inconsistent12,15. Unlike body composition and ApC, several studies3,10,11,16,17 have shown association with risk factors for CVD, providing information for empirically validated cutoff points to be proposed11,16,17, the musculoskeletal HRPF components do not present sufficient evidence of association with low back pain. Thus, international results18-20 based on normative scales for the classification of Flex and F/R have been used7, though there are efforts at national level21. The aims of this study were: a) verify the ability of Flex and F/R in screening adolescents with low back pain and b) to propose cutoff points for F/R and Flex.

MATERIAL AND METHODS

Population and sample

This cross-sectional school-based observational study was conducted with information of adolescents aged 10-17 years enrolled in the project called “Habitual Physical Activity and Associated Factors in Children and Adolescents at the Western Border of Rio Grande do Sul”. The project was reviewed and approved by the Ethics Research Committee of the Federal University of Pampa (CEP Registration: 001/2011).

Sample size was calculated according to the following criteria: a) population of 15,210 students at this age group from the School Census of Basic Education in the year 201022; b) prevalence of 50% because it is a study with multiple outcomes c) 95% confidence interval (CI 95%); d) sampling error of 3 percentage points; and, e) 20% increase to compensate for possible losses and refusals. With the adoption of these criteria, the need to evaluate 1,196 adolescents was estimated.

The sampling criterion used was probabilistic multiphase. For the selection of schools in the urban area, the city was divided into four regions, with regard to the proportion of students enrolled in schools in each region relative to the total number of students. It was necessary to select two schools in three regions and three schools in one region. For the selection of students from rural areas, a simple draw among all schools of this area was carried out, requiring the selection of only one school.

All students of the selected schools enrolled from the fifth grade of elementary school to the twelfth grade of high school were invited to participate. The inclusion criteria were: a) being enrolled in the public school network; and b) providing free and informed consent form signed by parents and express willingness to participate by signing the consent form for underage. Exclusion criteria were: a) carriers of physical or cognitive disability that could compromise the performance of any measurement; and b) being outside the age range proposed.

Data collection procedure

Once the ten schools were selected (nine urban and one rural), contact with directions for presentation of justifications, objectives and procedures was held. After institutional permits, dates were scheduled for data collection. The procedures took place during classes in two or three days as needed. On the first day, classes to be evaluated were visited to explain to students the study reasons, goals and procedures and delivery of the informed consent form and consent term with the following instructions: a) the reasons for receiving the document; b) the document should be handed to a responsible and that after reading it, should sign it authorizing the student’s participation and, c) to participate in the study, the document should be returned the next day. On the second day, data collections were held. For students that parents signed the consent form, but forgot to return the document, a third day of visit was scheduled so that these students had the opportunity to participate. All measurements were performed by a team of evaluators (teachers and physical education students), who were properly trained. Data collection took place between May and November 2011.

Variables and data collection instruments

For this study, variables gender (male or female), age (years), and musculoskeletal HRPF (F/R and Flex) were used. Low back pain (dependent variable) was estimated by applying the adapted instrument proposed by Sjoile et al.15. Adolescents answered the following question: “Have you ever had lower back pain or discomfort (at the side of the question, there was a figure indicating the lower back location)?” The possible answers were: Never; rarely; often and always. Categories never and rarely were grouped and considered as “no back pain” and categories often and always were grouped and considered as “low back pain”.

Musculoskeletal HRPF was considered through its components (independent variables) F/R and Flex. Measurement procedures were those suggested by Projeto Esporte Brasil21. Abdominal test in 1 minute for F/R and sit and reach test for Flex.

Data analysis

Descriptive statistics was initially used. The occurrence of low back pain has been described from absolute and relative frequencies and 95% confidence intervals (95%). F/R and Flex were described by average and standard deviation. The description of data was performed by sex and age. To identify possible differences in the frequency of low back pain between sexes and the average F/R and Flex, the Chi-square tests for heterogeneity, the t test for independent samples and one-way Analysis of Variance followed by post hoc Bonferroni test, respectively, were used.

To identify the screening capacity of musculoskeletal HRPF regarding the identification of adolescents more likely to report low back pain and to propose cutoff points for F/R and Flex, the Receiver Operating Characteristic curve (ROC) was used. This procedure requires a “reference” variable (dichotomous variable - in the case of this study low back pain - yes / no) and a “Test” variable (continuous variable - in the case of this study F/R and Flex). As comparative analyses indicated gender differences (p <0.05) for the frequency of low back pain and the average F/R and Flex values (p <0.05), the ROC curve analyses were stratified by sex. Comparing the analysis of average F/R and Flex values among ages, some differences (p <0.05) were identified. Therefore, variables of musculoskeletal HRPF (test variables) were standardized (Z score) for age and separated by sex for the ROC curve analysis.

With low back pain as reference (yes / no) and test variables defined, the areas under the ROC curve have been identified for each analysis. Those with lower limit of CI95% of the area under the curve higher than 50% were considered for the identification of the cut points. For this procedure, the specific Z score values of F/R and Flex in both sexes that presented the best fit between sensitivity and specificity were considered as the cutoff point. These values were used to identify the percentile value in each age group and sex corresponding to cutoff point found. All analyses were performed using SPSS software for Windows version 20.0.

RESULTS

Overall, 1,554 adolescents were selected to participate. Of these, 41 did not return the free and informed consent form signed by parents, were not present on the day of data collection and 50 refused to participate. Among those evaluated (1,463), eight were excluded because they are outside the age group of interest. Information on low back pain was reported by 1,377 adolescents, corresponding to 94.6% of the total sample (1,455). The frequency of low back pain was 16.1% (CI 95%: 14.2-18.0). In the analysis stratified by sex, the frequency was statistically higher (p <0.05) among girls (21.6%; CI 95%: 18.5-24.6) than among boys (10.5%; CI 95%: 7.8-12.4) (Figure 1).

Figure 1 Prevalence of low back pain among adolescents 

Table 1 presents the descriptive results of F/R and Flex throughout ages and separated by sex. F/R is higher in males at all ages (p <0.05). Although showing higher mean values at all ages for females, Flex showed statistically significant difference (p <0.05) only after the age of 11 years. Regarding the mean differences among ages, F/R is increasing in males (p <0.05), but these increases were not significant in subsequent ages. F/R for female and Flex for both sexes showed small variations of increase and decrease in mean values and there is no statistically significant difference among ages (p> 0.05).

Table 1 Means and standard deviations of muscle strength / resistance and flexibility of adolescents stratified by sex and age 

Male Female Male Female
Muscle strength / resistance Flexibility
Age (years) M SD M SD p M SD M SD p
10 32.04 12.45 26.80 8.57 0.033 21.84 7.32 23.27 6.48 0.368
11 33.51 8.97 27.50 9.69 <0.001 19.29 7.27 21.54 8.47 0.043
12 34.89 9.92 26.47 8.34 <0.001 19.02 8.50 20.90 7.84 0.104
13 38.61 8.32 28.32 9.04 <0.001 20.77 8.62 21.62 7.20 0.420
14 39.53 7.92 24.81 8.13 <0.001 22.56 7.80 23.08 7.95 0.641
15 39.49 9.57 24.55 8.61 <0.001 22.77 7.80 23.56 8.49 0.509
16 41.21 8.22 24.71 8.86 <0.001 25.42 8.03 24.38 7.47 0.404
17 39.71 11.83 23.37 7.85 <0.001 20.71 8.84 23.43 8.24 0.099

F/R: muscle strength / resistance; Flex: flexibility; M: mean; SD: standard deviation; p: p-value.

The analyses of the ROC curve were stratified by gender only. Table 02 presents the results of the area under the ROC curve between standard scores (Z scores) of F/R and Flex and low back pain. Among the tested combinations, the results indicated that only Flex in girls was able to predict low back pain (p <0.05).

Table 2 Area under the ROC curve between muscle strength / resistance, flexibility and frequency of back pain 

Male Female
n AURC CI95% p n AURC CI95% p
F/R 684 0.527 0.457-0.596 0.480 671 0.517 0.462-0.572 0.545
Flex 680 0.491 0.422-0.560 0.811 684 0.577 0.523-0.631 0.006

F/R: muscle strength / resistance; Flex: flexibility; n: number; AURC: area under the ROC curve; CI 95%: 95% confidence interval; p: level of significance.

The adjustment analysis between Flex sensitivity and specificity and back pain among girls indicated the value of 0.97 (standard score) as the best balance. This value corresponded to the 51st percentile of the standard score distribution. Identifying the value corresponding to this percentile in every age group, the cutoff values ranged from 22 to 24 centimeters (Table 3). The sensitivity and specificity values were 0.535 and 0.620, respectively (Figure 2).

Table 3 Cutoff points for flexibility measured by the sit and reach test for females according to age 

Age (years) CP (cm)
10 23
11 23
12 22
13 23
14 24
15 24
16 24
17 24

CP: cutoff point for flexibility measured by the sit and reach test.

Figure 2 Area under the ROC curve between flexibility and low back pain in girls 

DISCUSSION

This study is the first held in Brazil aimed at proposing cutoff points for F/R and Flex from the association of these variables with a health outcome using a representative sample of adolescents. The outcome used, low back pain, was chosen because theoretical assumptions indicate that low levels of both components of the musculoskeletal HRPF analyzed were associated with increased likelihood of low back pain complaints in adolescents1,4. The results of the ROC curve analysis indicated that Flex for girls was able to predict low back pain (p <0.05), confirming in part the hypothesis that adequate levels of this component are important for maintenance of musculoskeletal health in adolescence.

The frequency of low back pain among adolescents from Uruguaiana (16.1%) was lower than that of other studies15,23-26. However, the ways these investigations were conducted should be taken into account. Methodological aspects, such as the instruments used to estimate low back pain limited comparisons among results obtained. However, when the analysis is separated by gender, frequency of back pain is higher among girls in all studies, regardless of the instrument used. Culturally, women seem to show algesic conditions more frequently27; pain sensitivity is higher in women28; anatomically, women have a more fragile musculoskeletal condition29; and postural changes in the lumbar region are more frequent among female adolescents12. These features can help in understanding the higher frequency of low back pain in women.

The cutoff points found in this study for Flex among girls over the ages suggest that adolescents with levels below proposed values have increased likelihood of being affected by low back pain. These cutoff points were three to six centimeters above the health criteria proposed by Proesp-BR21, one and two centimeters below the health criteria proposed by the Physical Best18, and up to 5 centimeters below criteria proposed by the Fitnessgram19. It is noteworthy that currently Fitnessgram20adopts Back-Saver Sit and Reach test as one of the flexibility indicators of its battery of tests. This test presents some differences in implementation in relation to the sit and reach test. This impairs the comparison of the cutoffs values proposed for this test with those proposed for the sit and reach test.

The results of the analysis between Flex and low back pain in this study are in line with those reported by Lemos et al.12 and Sjolie et al.15. In these studies, the authors found an association between lower Flex levels and hyperlordosis12 and low back pain15 in adolescents. Together, these findings reinforce the importance of Flex as a health indicator of the column early in childhood and adolescence.

Although studies cited above have indicated association between lower Flex levels and presence of hyperlordosis and low back pain in adolescents, this study was the first to identify specific values for the screening of female adolescents with increased likelihood to show low back pain, which is the main contribution of this study. The proposition of cutoff points for field tests of musculoskeletal HRPF for this purpose is timely because laboratory methods for assessing Flex and F/R, when considering high population levels of measure, become unfeasible. Considering the ease of application, reduced time for measurement and low cost, the results of the sit and reach test as a way of measuring Flex enable this test and these cutoffs to be used as tools to identify adolescents more exposed to low back pain complaints.

Although it was possible to propose cutoff points for Flex from its association with low back pain among girls, this result was not observed for boys and for F/R in both sexes. The explanation for these results is impaired because other studies analyzing musculoskeletal HRPF and health outcomes from the ROC curve are not available. However, it is noteworthy that the study by Lemos et al.12 found association between lower F/R and Flex levels and increased likelihood of adolescents of both genders to show low back pain. Although the outcome used and statistical analyses have been different, the results indicate a possible association between musculoskeletal HRPF and spinal health of adolescents. In practical terms, an alternative for the evaluation of Flex in males and F/R in both sexes would be the use of cutoff points suggested by Proesp-BR21, which proposes these values from the results of an association study between components of musculoskeletal HRPF and postural changes in adolescents12. However, for this issue to be better understood, further studies are needed.

In addition to the features already presented that strengthen the present study, the sample size and the selection process of adolescents should be highlighted. These characteristics give accuracy and representativeness to the sample, increasing the power of analyses and the internal validity of the study. Despite the contributions provided by this study, some limitations and their possible repercussions must be discussed. The study design makes establishing cause and effect relationships impossible. The use of recall for the measurement of low back pain is a weakness, in which exhibiting symptoms and pain conditions can be a barrier found by young people. Another aspect concerns tests used for the measurement of musculoskeletal HRPF. Even with encouragement for adolescents to reach their best performance, the motivational aspect can influence the results.

CONCLUSIONS

Flex in females can be used as a tool for screening girls with increased likelihood to show low back pain complain. The cutoff points proposed in this study are important to approach and minimize the heterogeneity of literature results and an important support tool for the early identification of low back pain in adolescents and may help reverse the increasing incidence of its prevalence in this age group.

REFERENCES

1 Zhu W, Mahar MT, Welk GJ, Going SB, Cureton KJ. Approaches for Development of Criterion-Referenced Standards in Health-Related Youth Fitness Tests. Am J Prev Med 2011;41(4S2):S68-S76. [ Links ]

2 Wennlöf AH, Ruiz JR, Harro M, Sjöström M. Cardiorespiratory fitness relates more strongly than physical activity to cardiovascular disease risk factors in healthy children and adolescents: the European Youth Heart Study. J Cardiovasc Risk Reabil 2007;14(4):575-81. [ Links ]

3 Neto AE. Fatores de Risco para Aterosclerose Associados à Aptidão Cardiorrespiratória e ao IMC em Adolescentes. Arq Bras Endocrinol Metab 2008;52(6):1024-30. [ Links ]

4 ACSM. American College of Sports Medicine. Manual para teste de esforço e prescrição de exercício. 2010, 8ª. ed. Rio de Janeiro: Guanabara Koogan. [ Links ]

5 Bergmann GG, Bergmann MLA, Garlipp DC, Lorenzi TDC, Gaya ACA. Alteração anual no crescimento e na aptidão física relacionada à saúde de escolares. Rev Bras Cineantropom Desempenho Hum 2005;7(2):55-61. [ Links ]

6 Araújo SS, Oliveira ACC. Aptidão física em escolares de Aracajú. Rev Bras Cineantropom Desempenho Hum 2008;10(3):271-6. [ Links ]

7 Pelegrini A, Silva DAS, Petroski EL, Glaner MF. Aptidão física relacionada à saúde de escolares brasileiros: dados do projeto esporte Brasil. Rev Bras Med Esporte 2011;17(2):92-6. [ Links ]

8 Rodrigues LP, Angélico S, Saraiva L, Bezerra P. Estabilidade da aptidão física na transição da infância (7-9 anos) para a puberdade (15 anos): o Estudo Morfofuncional da Criança Vianense. Rev Port Cien Desp 2007;7(3):347-57. [ Links ]

9 Alves JGB, Montenegro FMU, Oliveira FA, Alves RV. Prática de esportes durante a adolescência e atividade física de lazer na vida adulta. Rev Bras Med Esporte 2005;11(5):291-4. [ Links ]

10 Bergmann MLA, Bergmann GG, Halpern R, Rech RR, Constanzi CB, Alli LR. Colesterol total e fatores associados: estudo de base escolar no sul do Brasil. Arq Bras Cardiol 2011;97(1):17-25. [ Links ]

11 Bergmann GG, Gaya ACA, Bergmann MLA, Halpern R, Rech RR, Constanzi CB, et al. Pontos de corte para a aptidão cardiorrespiratória e a triagem de fatores de risco para doenças cardiovasculares na infância. Rev Bras Med Esporte 2010;16(5):339-43. [ Links ]

12 Lemos AT, Santos FR. Gaya ACA. Hiperlordose lombar em crianças e adolescentes de uma escola privada no Sul do Brasil: ocorrência e fatores associados. Cad Saúde Pública 2012;28(4):781-8. [ Links ]

13 Hoy D, March L, Brooks P, Blyth F, Woolf A, Bain C, et al. The global burden of low back pain: estimates from the Global Burden of Disease 2010 study. Ann Rheum Dis 2014;73(6):968-974 [ Links ]

14 Balagué F, Dutoit G, Waldburguer M. Low back pain in schoolchildren – an epidemiological study. Scand J Rehabil Med 1988;20(4):175-9. [ Links ]

15 Sjölie AN. Low-back pain in adolescents is associated with poor hip mobility and high body mass index. Scand J Med Sci Sports 2004;14(3):168-75. [ Links ]

16 Bergmann GG, Gaya A, Halpern R, Bergmann MLA, Rech RR, Constanzi CB, et al. Índice de massa corporal para a triagem de fatores de risco para doenças cardiovasculares na infância. Arq Bras Endocrinol Metab 2011;55(2):114-20. [ Links ]

17 Welk GJ, Laurson KR, Eisenmann JC, Cureton KJ. Development of youth aerobic-capacity standards using receiver operating characteristic curves. Am J Prev Med 2011;41(4S2):S111-S116. [ Links ]

18 AAHPERD. Physical Best. American Alliance for Health, Physical Education and Recreation and Dance. Reston, Virginia, 1988. [ Links ]

19 Institute for Aerobic Research. Fitnessgram User’s Manual. Dallas, Texas, Institute for aerobics. Research, 1987. [ Links ]

20 Plowman SA, Meredith MD. Fitnessgram / Activitygram Reference Guide. 4th. Ed. Dallas, TX: The Cooper Institute, 2013; Available from:http://www.cooperinstitute.org [2014 dec 20]. [ Links ]

21 Projeto Esporte Brasil;Availablefrom:http://www2.ufrgs.br/proesp/ [2014 dec 17]. [ Links ]

22 Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira (Inep). Censo Escolar da Educação Básica; Availablefrom:http://portal.inep.gov.br/basica-censo [2010 oct 10]. [ Links ]

23 Harreby M, Nygaard B, Jessen T, Larsen E, Storr-Paulsen A, Lindahl A et al. Risk factors for low back pain in a cohort of 1389 Danish school children: an epidemiologic study. Eur Spine J 1999;8(6):444-50. [ Links ]

24 Szpalski M, Gunzburg R, Balague F, Nordin M, Melot CA. 2-year prospective longitudinal study on low back pain in primary school children. Eur Spine J 2002;11(5):459-64. [ Links ]

25 Masiero S, Carraro E, Celia A, Sarto D, Ermani M. Prevalence of nonspecific low back pain in schoolchildren aged between 13 and 15 years. Acta Paediatr 2002;97(2):212-6. [ Links ]

26 Noll M, Candotti CT, Tiggemann CL, Schoenell MCW, Vieira A. Prevalence of back pain and associated factors in elementary school students in the town of Teutônia, Rio Grande do Sul. Rev Bras Saude Mater Infant 2012;12(4):395-402. [ Links ]

27 De Vitta AL, Martinez MG, Piza NT, Simeão SF, Ferreira NP.Prevalência e fatores associados à dor lombar em escolares. Cad Saúde Pública 2011;27(8):1520-15288. [ Links ]

28 Quiton RL, Greenspan JD. Sex differences in endogenous pain modulation by distracting and painful conditioning stimulation. Pain 2007;132(Suppl 1):S134-S149. [ Links ]

29 Silva MC, Fassa AG, Valle NC. Dor lombar crônica em uma população adulta do Sul do Brasil: prevalência e fatores associados. Cad Saúde Pública 2004;20(2):377-85. [ Links ]

Received: July 26, 2015; Accepted: November 17, 2015

Gabriel Gustavo Bergmann, BR 472 - Km 592 - Caixa Postal 118, CEP: 97508-000 - Uruguaiana – RS. Brasil. E-mail: gabrielbergmann@unipampa.edu.br

Creative Commons License This is an Open Access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.