Validation of the Brazilian version of the World Health Organization Disability Assessment Schedule in individuals with diabetes mellitus

Validação da versão brasileira do World Health Organization Disability Assessment Schedule em indivíduos com diabetes mellitus

Validación de la versión brasileña del World Health Organization Disability Assessment Schedule en personas con diabetes mellitus

ABSTRACT

To evaluate the functioning of individuals with diabetes mellitus (DM) using a biopsychosocial model, recommended by the World Health Organization and sustained in the theoretical-conceptual framework of the International Classification of Functioning, Disability and Health (ICF), this study proposed validating the Brazilian version of WHODAS 2.0 (World Health Organization Disability Assessment Schedule). The 36-item version of WHODAS 2.0 was applied to 100 participants with DM as validation procedures, using the measurement of Diabetes Quality of Life Measure (DQOL-Brazil) and a dynamometry. The psychometric properties analyzed were internal consistency (Cronbach Alpha coefficient) and convergent and divergent external validity (Spearman correlation coefficient). The internal consistency analysis was appropriate, except for the “having a good relationship with people” domain in the alpha Cronbach coefficient. External validity analysis confirmed the convergence hypothesis between the correlate domains of the different tools. The exception was the “life activities” domain (WHODAS) with the left-hand dynamometry. The Brazilian version of the WHODAS 2.0 instrument is a valid instrument to assess the functioning of these individuals.

Keywords|
Diabetes Mellitus; Validation Studies; International Classification of Functioning, Disability and Health

RESUMO

Visando avaliar a funcionalidade dos sujeitos com diabetes mellitus (DM) usando um modelo biopsicossocial, recomendado pela Organização Mundial de Saúde, e ancorado no arcabouço teórico-conceitual da Classificação Internacional de Funcionalidade, Incapacidade e Saúde, este estudo propôs a validação da versão brasileira do WHODAS 2.0 (World Health Organization Disability Assessment Schedule). A versão de 36 itens do WHODAS 2.0 foi aplicada a 100 sujeitos com DM. Como instrumentos auxiliares à validação, utilizou-se o instrumento Diabetes Quality of Life Measure (DQOL-Brasil) e a dinamometria. As propriedades psicométricas analisadas foram consistência interna (coeficiente Alfa de Cronbach) e validade externa - convergente e divergente (coeficiente de correlação de Spearman). A análise de consistência interna mostrou-se apropriada, à exceção do domínio “relações interpessoais”. A análise da validade externa confirmou as hipóteses de convergência esperadas na comparação dos domínios correlatos dos instrumentos auxiliares utilizados no processo de validação, exceto no domínio “atividades de vida” (WHODAS) com a dinamometria da mão esquerda. Conclui-se que a versão brasileira do instrumento WHODAS 2.0 é válida para aferição da funcionalidade nesses indivíduos.

Descritores|
Diabetes Mellitus; Estudos de Validação; Classificação Internacional de Funcionalidade, Incapacidade e Saúde

RESUMEN

Para evaluar las limitaciones funcionales de las personas con diabetes mellitus (DM) utilizando un modelo biopsicosocial, recomendado por la Organización Mundial de la Salud y anclado en el marco teórico-conceptual de la Clasificación Internacional del Funcionamiento, de la Discapacidad y de la Salud, este estudio propuso validar la versión brasileña de WHODAS 2.0 (World Health Organization Disability Assessment Schedule). La versión de 36 ítems de WHODAS 2.0 se aplicó a 100 participantes con DM. Como procedimientos auxiliares de validación, se utilizaron la Diabetes Quality of Life Measure (DQOL-Brasil) y la dinamometría. Las propiedades psicométricas analizadas fueron la consistencia interna (coeficiente alfa de Cronbach) y la validez externa convergente y divergente (coeficiente de correlación de Spearman). El análisis de la consistencia interna fue apropiado, excepto en el dominio “llevarse bien con las personas” en el coeficiente alfa de Cronbach. El análisis de validez externa confirmó la hipótesis de convergencia entre los dominios correlacionados de las diferentes herramientas. La excepción fue el dominio “actividades de la vida” (WHODAS) con la dinamometría izquierda. Se concluye que la versión brasileña del WHODAS 2.0 es un instrumento válido para evaluar el funcionamiento de estos individuos.

Palabras clave|
Diabetes Mellitus; Estudios de Validación; Clasificación Internacional del Funcionamiento, de la Discapacidad y de la Salud

INTRODUCTION

Diabetes mellitus (DM) is a group of genetic or acquired metabolic disturbances with a main characteristic - frequent hyperglycemia due to impaired insulin secretion and/or action^{1), (2}. DM is a chronic and incapacitating disease². These metabolic alterations cause persistent high glycemic levels, which can lead to variations in the functioning of individuals. Some of these variations are: difficulties when walking 400 meters³; climbing a ladder; performing house chores; and poor performance^{4), (5} when compared with same-age-non-diabetic individuals in physical evaluation measures (walking speed^{4), (5}, muscular strength^{4), (6}, sitting³ and getting up from a chair^{4), (7}). In general, these functioning impacts are consequences of the diabetic patient comorbidities^{3)- (5}.

Due to progressive increase in chronic diseases incidence, the study and evaluation of their impact in human functioning has academic and clinical importance¹. The World Health Organization (WHO) is constantly developing tools and models to rate disability, incapacity, and functioning. One of these tools is the International Classification of Functioning, Disability and Health (ICF), which establishes functioning in a multidimensional view. Its biopsychosocial definition includes functions and structures of the body, activity, participation as well as environmental and personal factors⁸.

The WHO also developed the World Health Organization Disability Assessment Schedule (WHODAS 2.0), a generic and practical questionnaire to assess health and disability in a populational or clinic scale. It allows designing and monitoring of the health interventions impact⁹. WHODAS 2.0 was developed based on the theoretical concept of the ICF and is able to quantify the patient’s functioning through analysis of cognition, mobility, self-care, good relationship with people, life activities and participation⁹.

According to health professionals who work with diabetes patients, the assessment of a functioning tool based on the ICF will allow the design of more appropriate intervention and assessment strategies to the biopsychosocial model. Furthermore, diabetes patients will benefit from more effective and broadly based interventions and the health system as a whole could plan the offering of interprofessional services according to the biopsychosocial model.

The objective of this study was to validate the Brazilian version of the WHODAS 2.0 tool via analysis of its psychometric properties to assess functioning of DM patients.

METHODOLOGY

Participants

One hundred patients who had DM diagnosis and follow-up by an endocrinologist were included in this study. They were selected by a convenience sample among people waiting for a health appointment at the Universidade Federal do Triângulo Mineiro (UFTM) specialties ambulatory or at a primary health unit in Uberaba. The following inclusion criteria were used: confirmed medical diagnosis of DM; currently being treated or supervised in an ambulatory; age equal or above 18 years old; consent to participate in the study; and signature of the free and informed term of consent. People who could not understand or answer the questions and those who had physical disabilities unrelated to DM were excluded. These criteria were confirmed on the patients’ medical records.

Instruments and admeasurement tools

The following tools were used in this study: World Health Organization Disability Assessment Schedule 2.0 (WHODAS 2.0) and Diabetes Quality of Life Measure (DQOL-Brazil). The latter was selected since its method is similar to those studied in WHODAS.

WHODAS 2.0 is a WHO-developed generic tool used to assess functioning and health of any individual regardless of their health condition¹⁰. It consists of 36 questions covering six domains: cognition (6 items); mobility (5 items); self-care (4 items); good relationship with people (5 items); life activities (8 items); and participation (8 items). Each question has five alternatives as possible answers; the first answer represents “No struggle” and the fifth answer means “Extreme struggle or unable to perform”. The sum of the answers constitutes a 0-100 score in which the higher the number, the worse the functioning level. WHODAS questionnaire is available in three different versions according to the number of questions: 36, 12 and a hybrid version with 12+24 questions. It also varies in method of administration (self-administered, interview or proxy-administered). In this study, the complete 36-question questionnaire was applied through interview by a trained interviewer⁹. WHODAS 2.0 is translated and validated to Brazilian Portuguese¹⁰.

DQOL-Brazil was developed specifically for diabetes patients in order to assess quality of life¹¹. It is also validated to be used in Brazil¹². The Brazilian version is made of 44 items distributed in four domains: satisfaction with treatment (15 items); impact of treatment (18 items); concern about social/vocational issues (7 items); and concern about the future effects of diabetes (4 items). DQOL-Brazil uses the Likert scale of answers which varies from 1 to 5 according to the level of satisfaction reported by the interviewee. The final score is obtained by the arithmetic mean of individual answers, in which the lower the score, the better the participant’s quality of life¹².

Both WHODAS and DQOL-Brazil questionnaires were conducted at the same time by an interview performed in a private room of the UFTM ambulatory or in a primary health care unit in Uberaba. In the same occasion, social and demographic data, anthropometric measures and comorbidities information were also collected from the participants.

In this study, we also assess the strength of both hands using a dynamometer. This tool is recommended by the American Society of Hand Therapists (ASHT) ⁽¹³ and is certified to estimate the global muscular strength through the hand grip technique^{14), (15}. Participants were sitting in an armless chair, with both feet on the ground and with the hip and knee joints flexed at approximately 90 degrees. The shoulder at the same side of the examined limb was adduced in neutral rotation and the elbow was flexed at 90 degrees with the forearm in neutral position. The hand that was not being tested was left resting on the ipsilateral thigh. Participants were instructed to keep a steady position during tests and were corrected when necessary. The final variable was determined by the arithmetic mean of three measurements in each hand¹⁴.

Statistical analysis

The reliability was assessed by the internal consistence of the tool, using the alpha coefficient of Cronbach. To assess the (convergent and divergent) external validity, the Spearman correlation coefficient was used. As a priori assumption, we established the (convergent/divergent) relational hypothesis between the WHODAS questionnaire, DQOL-Brazil measures/domains, and dynamometry, as showed in Table 1.

Table 1
Relational hypothesis between WHODAS domains and the other methods used

WHODAS/Domain	DQOL/Domain	Dynamometry	Expected relation
Mobility		Dynamometry	Convergent
Mobility	Concern about social/vocational issues		Divergent
Self-care	Concern about future effects of diabetes	Dynamometry	Convergent
Having a good relationship with people		Dynamometry	Divergent
Having a good relationship with people	Satisfaction with treatment		Convergent
Life activities		Dynamometry	Convergent
Participation	Satisfaction with treatment		Convergent
Participation	Concern about social/vocational issues		Convergent
Total	Satisfaction with treatment	Dynamometry	Convergent

Moderate correlations were expected since the comparable tools have similar but not identical domains, a fact that excludes the possibility of strong correlations. Here, we consider correlations with coefficients between 0.10 and 0.39 as weak, coefficients between 0.40 and 0.69 as moderate and coefficients between 0.70 and 1¹⁶⁾ as strong.

All statistical analysis was described and executed using the Stata 13 software package. A statistical significance level of 5% was considered.

RESULTS

Sample characterization

All 100 participants properly answered the questions. Tables 2 and 3 show the sample description.

Table 2
General sample characterization

Variables		n (100)	% (100)
Gender
	Women	73	73
	Men	27	27
Skin Color
	White	59	59
	Brown	12	12
	Black	29	29
Education
	Illiteracy	4	4.04
	Some primary school	52	52.53
	Primary school	18	18.18
	Some high school	3	3.03
	High school	13	13.13
	Technical education	3	3.03
	Some graduation	1	1.01
	Graduation	5	5.05
Marital status
	Single	14	14
	Married	53	53
	Separated	4	4
	Divorced	12	12
	Widow	13	13
	Live together	4	4
Work
	Paid work	28	28
	Freelancer	9	9
	Unpaid work	1	1
	Homemaker	17	17
	Retired	31	31
	Unemployed	3	3
	Other	11	11
Body mass index
	Underweight (<18.5)	1	1
	Healthy weight (18.5-24.9)	27	27
	Overweight (25-29.9)	37	37
	Obese Class I (30-34.9)	20	20
	Obese Class II (35-39.9)	7	7
	Obese Class III (>40)	8	8
	Insulin dependent	43	43
Variables		Mean	Standard deviation
	Age (years)	54.75	12.35
	Diagnose time (years)	10.82	8.67
WHODAS
	Cognition	58.35	26.93
	Mobility	64.37	33.50
	Self-care	56.10	27.66
	Having a good relationship with people	58.33	21.38
	Life activities	24.36	12.58
	Participation	68.50	28.66
	Total	56.25	19.91
DQOL
	Satisfaction with treatment	2.26	0.71
	Impact of treatment	2.19	0.68
	Concern about social/vocational issues	1.43	0.69
	Concern about the future effects of diabetes	2.54	0.97
	Left hand dynamometry	24.61	9.70
	Right hand dynamometry	26.12	9.74

Table 3
Sample characterization of diabetes mellitus comorbidities

Diseases	n	%
None	31	31
Amputation	2	2
Sensibility alterations	43	43
Visual alterations	48	48
Kidney diseases	15	15
Cardiovascular diseases	1	1
Wounds	2	2
Others	3	3
Total	145*	100

• *Some people have more than one disease

Psychometric properties

The data of the 36 items of this questionnaire were verified in each of its domains. Table 4 shows results of internal consistency (IC).

Table 4
WHODAS Alpha Cronbach distribution (Internal consistency)

WHODAS 2.0 - domains	Cronbach α
Cognition	0.84
Mobility	0.85
Self-care	0.72
Good relationship with people	0.49/0.581
Life Activities	0.91
Participation	0.84
Total	0.94

• ¹
  Excluding the sexual relations question.

Using the alpha test of Cronbach in each domain, results that varied between 0.49 and 0.91 were obtained. The total score obtained a Cronbach alpha of 0.94. This shows that the tool has a suitable internal consistency, that is, values between 0.70 and 0.90¹⁷ in five of its six domains (the Good relationship with people domain was the exception).

In Table 5, to validate the convergent and divergent criteria, correlations of the WHODAS 2.0 domains with the DQOL-Brazil domains were verified. It showed that, in general, the questionnaire presented moderate correlation.

Table 5
WHODAS 2.0 correlation with DQOL-Brazil

	WHODAS/Domains
	Cognition	Mobility	Self-care	Good relationship with people	Life activities	Participation	Total
DQOL - Brazil
Satisfaction with treatment	0.5358*	0.4204*	0.3016*	0.4489*	0.3567*	0.6582*	0.6332*
Impact of treatment	0.5308*	0.4703*	0.3499*	0.4216*	0.3805*	0.7202*	0.6577*
Concern about the future effects of diabetes	0.2891*	0.0709	0.1468	0.2745*	0.1941	0.2013*	0.2560*
Concern about social/vocational issues	0.5127*	0.3788*	0.3042*	0.4124*	0.3814*	0.5896*	0.5738*
Left hand dynamometry	−0.4143*	−0.4059*	−0.3045*	−0.0824	−0.1252	−0.2916*	−0.3444*
Right hand dynamometry	−0.3905*	−0.4298*	−0.3151*	−0.0078	−0.2190*	−0.3494*	−0.3948*

• *p<0.05 (Spearman correlation test).

DISCUSSION

The WHODAS questionnaire showed suitable internal consistency to be used with DM patients in almost all domains, as other studies with similar Cronbach alpha coefficients have shown^{18), (19}. The only exception was in the Good relationship with people domain, in which, as other studies demonstrate^{20), (21), (22}, the sexual activities question decreased the internal consistency of the domains. Therefore, the use of this question is debatable, as its advantages and disadvantages should be considered for the questionnaire’s internal consistency. Some studies may assess functioning in different ways and, for some of them, the sexual activities item might not be important, but we emphasize that disturbance in sexual activities is frequently noticed^{23), (24), (25}, which shows the importance of this subject regarding the health and functioning of DM patients.

Analysis of external validity confirmed all convergence hypotheses presented in this study, except for the Life activities domain (WHODAS) with left hand dynamometry (DQOL-Brazil). The two divergent relations were also confirmed by the conducted statistical analysis, which shows that the tool has external validity to be used in people with diabetes.

Therefore, corroborating other studies that used the complete version of WHODAS 2.0, the validity of the Brazilian version of the WHODAS 2.0 questionnaire shows satisfactory psychometric properties in general, which allows its use to assess functioning in DM patients^{19), (26}.

A limitation of this study is the geographic restriction, since each locality may have its own specificities despite dealing with the same health condition. On the other hand, this study provides a generic tool based on ICF to assess functioning of a specific group. The WHODAS 2.0 validity allows health professionals to use it to assess functioning in a broader way according to WHO recommendations.

CONCLUSION

WHODAS 2.0 instrument is valid and reliable to assess DM patient functioning. Cautious use of this questionnaire is suggested, specifically when asking about sexual activities in the Good relationship with people domain. More discussion is necessary on the use of this question.

REFERENCES

• ¹
  International Diabetes Federation. IDF Diabetes Atlas. 8th ed. Brussels; 2017 [cited 2019 Sept 19]. Available from: http://www.diabetesatlas.org
  » http://www.diabetesatlas.org
• ²
  Sociedade Brasileira de Diabetes. Diretrizes da Sociedade Brasileira de Diabetes 2017-2018. São Paulo: Clannad; 2017 [cited 2019 Sept 19]. Available from: https://www.diabetes.org.br/profissionais/images/2017/diretrizes/diretrizes-sbd-2017-2018.pdf
• ³
  Ryerson B, Tierney EF, Thompson TJ, Engelgau MM, Wang J, Gregg EW, Geiss LS. Excess physical limitations among adults with diabetes in the U.S. population, 1997-1999. Diabetes Care. 2003;26(1):206-10. doi: 10.2337/diacare.26.1.206
  » https://doi.org/10.2337/diacare.26.1.206
• ⁴
  Gregg EW, Beckles GL, Williamson DF, Leveille SG, Langlois JA, Engelgau MM, Narayan KM. Diabetes and physical disability among older U.S. adults. Diabetes Care. 2000;23(9):1272-7. doi: 10.2337/diacare.23.9.1272
  » https://doi.org/10.2337/diacare.23.9.1272
• ⁵
  Volpato S, Blaum C, Resnick H, Ferrucci L, Fried LP, Guralnik JM, Women's Health and Aging Study. Comorbidities and impairments explaining the association between diabetes and lower extremity disability: the women's health and aging study. Diabetes Care. 2002;25(4):678-83. doi: 10.2337/diacare.25.4.678
  » https://doi.org/10.2337/diacare.25.4.678
• ⁶
  Sinclair AJ, Conroy SP, Bayer AJ. Impact of diabetes on physical function in older people. Diabetes Care. 2008;31(2):233-5. doi: 10.2337/dc07-1784
  » https://doi.org/10.2337/dc07-1784
• ⁷
  De Rekeneire N, Resnick HE, Schwartz AV, Shorr RI, Kuller LH, Simonsick EM, et al. Diabetes is associated with subclinical functional limitation in nondisabled older individuals: the health, aging, and body composition study. Diabetes Care. 2003;26(12):3257-63. doi: 10.2337/diacare.26.12.3257
  » https://doi.org/10.2337/diacare.26.12.3257
• ⁸
  World Health Organization. International classification of functioning, disability and health (ICF). Geneva; 2001 [cited 2019 Sept 19]. Available from: https://www.who.int/classifications/icf/en/
  » https://www.who.int/classifications/icf/en/
• ⁹
  Ustün TB, Chatterji S, Kostanjsek N, Rehm J, Kennedy C, Epping-Jordan J, et al. Developing the World Health Organization Disability Assessment Schedule 2.0. Bulletin of the World Health Organization. 2010;88:815-23. doi: 10.2471/BLT.09.067231
  » https://doi.org/10.2471/BLT.09.067231
• ¹⁰
  Castro SS, Leite C F. Avaliação de saúde e deficiência: manual do WHO Disability Assessment Schedule (WHODAS 2.0). Uberaba: Organização Mundial da Saúde; 2015 [cited 2019 Sept 19]. Available from: http://apps.who.int/iris/bitstream/10665/43974/19/9788562599514_por.pdf
• ¹¹
  Reliability and validity of a diabetes quality-of-life measure for the diabetes control and complications trial (DCCT). Diabetes Care. 1988;11(9):725-32. doi: 10.2337/diacare.11.9.72
  » https://doi.org/10.2337/diacare.11.9.72
• ¹²
  Correr CJ, Pontarolo R, Melchiors AC, Rossignoli P, Fernández-Llimós F, Radominski RB. Tradução para o português e validação do instrumento Diabetes Quality of Life Measure (DQOL-Brasil). Arq Bras Endrocrinol Metab. 2008;3(52):515-22. doi: 10.1590/S0004-27302008000300012
  » https://doi.org/10.1590/S0004-27302008000300012
• ¹³
  American Society of Hand Therapists. Clinical assessment recommendations. 2nd ed. Chicago: The Society; 1992.
• ¹⁴
  Reis MM, Arantes PMM. Assessment of hand grip strength-validity and reliability of the saehan dynamometer. Fisioter Pesqui. 2011;18(2):176-81. doi: 10.1590/S1809-29502011000200013
  » https://doi.org/10.1590/S1809-29502011000200013
• ¹⁵
  Incel NA, Ceceli E, Durukan PB, Erdem HR, Yorgancioglu ZR. Grip strength: effect of hand dominance. Singapore Med J. 2002;43(5):234-7.
• ¹⁶
  Dancey CP, Reidy J. Estatística sem matemática para psicologia. 5th ed. Porto Alegre: Penso; 2013.
• ¹⁷
  Bland JM, Altman DG. Statistics notes: Cronbach's alpha. BMJ. 1997;314:572. doi: 10.1136/bmj.314.7080.572
  » https://doi.org/10.1136/bmj.314.7080.572
• ¹⁸
  Meesters JJ, Verhoef J, Liem IS, Putter H, Vliet Vlieland TP. Validity and responsiveness of the World Health Organization Disability Assessment Schedule II to assess disability in rheumatoid arthritis patients. Rheumatology (Oxford). 2010;49(2):326-33. doi: 10.1093/rheumatology/kep369
  » https://doi.org/10.1093/rheumatology/kep369
• ¹⁹
  Garin O, Ayuso-Mateos JL, Almansa J, Nieto M, Chatterji S, Vilagut G, et al. Validation of the "World Health Organization Disability Assessment Schedule, WHODAS-2" in patients with chronic diseases. Health Qual Life Outcomes. 2010;8:51. doi: 10.1186/1477-7525-8-51
  » https://doi.org/10.1186/1477-7525-8-51
• ²⁰
  Magistrale G, Pisani V, Argento O, Incerti CC, Bozzali M, Cadavid D, et al. Validation of the World Health Organization Disability Assessment Schedule II (WHODAS-II) in patients with multiple sclerosis. Mult Scler. 2015;21(4):448-456. doi: 10.1177/1352458514543732
  » https://doi.org/10.1177/1352458514543732
• ²¹
  Hu L, Zang YL, Li N. The applicability of WHODAS 2.0 in adolescents in China. J Clin Nurs. 2012;21(17-18):2438-51. doi: 10.1111/j.1365-2702.2012.04126
  » https://doi.org/10.1111/j.1365-2702.2012.04126
• ²²
  Kutlay S, Küçükdeveci AA, Elhan AH, Oztuna D, Koç N, Tennant A. Validation of the World Health Organization disability assessment schedule II (WHODAS-II) in patients with osteoarthritis. Rheumatol Int. 2011;31(3):339-46. doi: 10.1007/s00296-009-1306-8
  » https://doi.org/10.1007/s00296-009-1306-8
• ²³
  Schiavi RC, Stimmel BB, Mandeli J, Rayfield EJ. Diabetes mellitus and male sexual function: a controlled study. Diabetologia. 1993;36(8):745-51. doi: 10.1007/bf00401146
  » https://doi.org/10.1007/bf00401146
• ²⁴
  Copeland KL, Brown JS, Creasman JM, Van Den Eeden SK, Subak LL, Thom DH, et al. Diabetes mellitus and sexual function in middle-aged and older women. Obstet Gynecol. 2012;120(2Pt1):331-40. doi: 10.1097/AOG.0b013e31825ec5fa
  » https://doi.org/10.1097/AOG.0b013e31825ec5fa
• ²⁵
  Giraldi A, Kristensen E. Sexual dysfunction in women with diabetes mellitus. J Sex Res. 2010;47(2):199-211. doi: 10.1080/00224491003632834
  » https://doi.org/10.1080/00224491003632834
• ²⁶
  Chiu TY, Yen CF, Chou CH, Lin JD, Hwang AW, Liao HF, Chi WC. Development of traditional Chinese version of World Health Organization Disability Assessment Schedule 2.0 36-item (WHODAS 2.0) in Taiwan: validity and reliability analyses. Res Dev Disabil. 2014;35(11):2812-20. doi: 10.1016/j.ridd.2014.07.009
  » https://doi.org/10.1016/j.ridd.2014.07.009

• Finance source: nothing to declare
• 3
  Approved by the Comitê de Ética da Universidade Federal do Triângulo Mineiro Protocol No. 948.994 on February 3 2015.

Publication Dates

• Publication in this collection
  02 Dec 2019
• Date of issue
  Oct-Dec 2019

History

• Received
  24 Oct 2018
• Accepted
  02 Aug 2019