A validated instrument to assess dentists’ knowledge about diabetes: the dental -diabetes questionnaire

Abstract

Aim

To elaborate and validate an instrument for Brazilian Portuguese speakers, to assess dentists’ knowledge about care of patients with diabetes mellitus (Dental-Diabetes).

Methods

Methodological study comprising four stages: a) Elaboration of instrument; b) Content validation (computing Content Validity Index - CVI) based on Expert Committee assessment; c) Pre-test with 30 dentists, followed by assessment of suggestions by Expert Committee; d) Psychometric validation through instrument application in a sample of 127 dentists by means of the web tool e-Surv. Cronbach’s alpha and intraclass correlation coefficients were used to evaluate, respectively, internal consistency and reproducibility.

Results

The final version of the instrument consists of 22 questions (7 on sociodemographic data and 15 querying dentists’ knowledge) and those submitted for validation attained a CVI of 0.95 [95% CI 0.916-0,981], showing satisfactory internal consistency, with 0.794 Cronbach’s alpha [95% CI 0.741-0.842] and an intraclass correlation coefficient of 0.799 [95% CI: 0.746-0.846] between the test and retest scores.

Conclusions

Dental-Diabetes is a comprehensive instrument, culturally adequate and validated to assess dentists’ knowledge about care of patients with diabetes.

Diabetes mellitus; Dentists; Knowledge; Validation studies as topic

Introduction

Diabetes Mellitus (DM), a chronic disease, is a public health problem that affects a large number of individuals from all social backgrounds. In 2019, the International Diabetes Federation (International Diabetes Federation, IDF) estimated that diabetes affected 9.3% of the world population, with an estimated prevalence of more than 10.9% people in 2045¹. If uncontrolled, systemic complications of DM can include heart attack, kidney disease, limb loss, blindness, and peripheral nerve damage².

Due to the complexity of diabetes, how well the disease is controlled is an important issue in dental treatment planning. Patients with DM manifest a high prevalence of oral problems such as periodontal disease, tooth loss, xerostomia, caries, burning mouth disorder, taste and salivary gland dysfunction, delayed wound healing, lichen planus, geographic tongue, and candidiasis³.

Being a common disease in dental practice, dentists are expected to be part of a multidisciplinary team, collaborating especially with endocrinologists. They are expected to base their care on strategies to provide effective management of DM and its oral consequences, identifying oral disease impacting glycemic control, which, in turn, can impact oral health⁴. They are also expected to be aware of the pathophysiology of DM, its oral manifestations, signs and symptoms, how to react in case of an emergency, risks involved, systemic repercussions of the use of medications and anesthetics, which can all add up to provide better care for patients⁵-⁷.

Assessing dentists’ conduct regarding DM is crucial to understand their knowledge, contributing to establishing targets for their training in public and private services and teaching institutions and defining guidelines for educational content and contributing to better care for patients with DM. Given the importance of dentists’ role and the lack of tools to assess their knowledge about DM, an instrument was felt to be needed to gather information about dentists’ knowledge and promote their education on related topics. A specific instrument for this purpose is not currently available. The aim of the study was to develop and validate an instrument for the assessment of dentists’ knowledge about diabetes (Dental-Diabetes).

Materials and Methods

This is a methodological and exploratory study carried out from January 2017 to August 2018 in the city of Belo Horizonte, in the State of Minas Gerais, Brazil. The project was approved by the Ethics and Research Committee Involving Human Beings (CAAE number 65656117.6.1001.5138) at Santa Casa of Belo Horizonte Hospital. Agreement to participate in the study was obtained by using a Free Informed Consent Form signed by participants when accessing an electronic questionnaire by means of the webtool e-Surv. An Expert Committee made up of five dentists, a nurse, an endocrinologist, a linguist and a statistician took part in elaboration of the instrument and assessed all stages until its final version (Figure 1).

Figure 1
Stages in the instrument’s elaboration.

STAGE 1 - Instrument’s development

Prior to elaboration of the instrument, three authors (M.A.G.M.G., A.A.O.P., J.S.R.) conducted a literature review in PubMed (U.S. National Library of Medicine), LILACS (Latin American and Caribbean Literature in Health Sciences databases), and SciELO (Scientific Electronic Library Online) databases to obtain state-of-the-art information about diabetes, national and international recommendations on dental treatment of people with diabetes and questionnaires used to assess professionals’ knowledge about a particular disease. The descriptors used for the queries were ‘Diabetes mellitus’, ‘Dentists’, ‘Knowledge’, and ‘validation studies’. National and international publications yielded by the database queries were screened²,⁵-¹³. Based on the gathered insights we decided to elaborate an instrument in the form of a questionnaire.

In establishing a general conceptual structure dentistry-endocrinology interface, our instrument was developed in two parts: the first section focusing on dentists’ sociodemographic profile (7 questions); and the second section aimed at assessing dentists’ knowledge about key aspects of DM and related care expected to be performed as part of their work (16 questions) (Version 1-V1).

STAGE 2 - Content validation

For content validation, a web address to access a web assessment form was sent by e-mail to 22 dentists, six endocrinologists and five linguists (Judges’ Committee), who evaluated each item of the instrument’s first version (V1). Criteria for participation in the Committee were either to be a professional dentist with or without clinical practice implicating diabetes (dentist profile); or have taken part in questionnaire elaboration or translation in the healthcare area (linguist profile); or to have clinical practice in diabetes (endocrinologist profile). Judges were selected based on their curriculum vitae.

The Judges’ Committee assessed clarity and relevance of each item in V1 and rated them with the following options: one star standing for need for full reformulation; two stars, partial reformulation (substantial revision needed); three stars, need for partial reformulation, with minor editing to enhance text style; and four stars in case of no need for reformulation. A comment box was also provided for the experts’ remarks and suggestions.

Once the evaluation was completed, the Content Validity Index (CVI – the level of agreement of experts on adequacy of the items) was computed: number of scores 3 and 4 divided by total number of scores by all Committee members. CVI indicates the degree to which a scale has an adequate sample of items to represent a construct of interest - that is, whether a domain of content for the construct is adequately represented by the items. Results higher than or equal to 0.78 are considered acceptable¹⁴. Upon computing CVI (higher than 0.78) and implementing the Committee’s suggestions to improve, V2 was obtained. V2 is substantially similar to V1, except for minor editing and spelling correction.

STAGE 3 – Pre-test

The author (M.A.G.M.G.) carried out the pre-test through face-to-face interviews with 20 dentists¹⁵,¹⁶ in 10 meetings following participants’ schedules - 2 group meetings with 5 dentists, 2 pair meetings and 6 individual meetings. First, the whole instrument was read by each participant individually; secondly, items were discussed to ensure whether they were clear, accurate, relevant and adequately arranged. Participant’s feedback was then assessed by the Expert Committee, who considered all relevant comments and redrafted those items that obtained less than 80% agreement¹⁴. Version 3 (V3) was thus obtained and tested on a newly selected group of 10 dentists (in 8 individual meetings and 2 in pair meetings).

STAGE 4 - Validation

A web address to access V3 in digital format on the e-Surv platform was sent via e-mail to 127 dentists selected by convenience from both public and private services and universities. The sample size was adequate considering a level of significance equal to 5%, test power equal to 80%, standard deviations equal to the test and retest scores and a correlation coefficient equal to 0.30 (minimum value detected in the consistency assessment). A minimum sample size requirement was 85 professionals. Retest was performed with those 127 dentists with a minimum interval of 7 days and a maximum of 21 days between the tests (average 16 days)¹⁷.

Statistical analysis

Absolute and relative frequencies were used to describe the sample characteristics and the proportion of correct answers to the instrument items. Internal consistency and reproducibility were verified to analyze the reliability of the construct. Cronbach’s alpha (CA) was used to assess the internal consistency of the instrument. Internal consistency is an assessment of whether items intended to measure the same construct produce similar scores. A high degree of internal consistency indicates that items meant to assess the same construct yield similar scores. There are a variety of internal consistency measures. Usually, they involve determining how highly these items are correlated and how well they predict each other. Cronbach’s alpha is a commonly used measure.

The instrument’s reproducibility was evaluated through test-retest (temporal stability), computing the intraclass correlation coefficient (ICC). The Kappa index was added for reproducibility and refers to the percentage of concordant responses in the test and retest, defined as the ratio between the number of individuals who selected the same answer (regardless of being correct or incorrect) at both test and retest and the total number of individuals¹⁶,¹⁸. Floor and ceiling effects were measured by the number of respondents receiving the minimum and maximum scores, respectively. The significance level adopted for the statistical tests was 5%. For data analysis, SPSS version 20.0 was used.

Results

Instrument development and Content validation

The development spanned 6 months. V1 of the instrument consisted of 23 questions. After reviewing V1 following the experts’ suggestions, a second version (V2) was obtained, with 23 questions. The instrument achieved a good score by the committee regarding clarity and relevance, with a total CVI of 0.95 [95% CI 0.916-0,981].

Pre-test

In face-to-face tests, which lasted 3 months, 30 dentists participated. 67% were female; 47% had a Diploma course and were working in different areas such as surgery, dentistry, endodontics, periodontics and others; 47% had more than 20 years’ experience; 100% reported having provided dental care to patients with diabetes; 67% declared not having had any training to treat patients with diabetes; however, 63% reported feeling empowered to provide care for Diabetes patients. (Table 1).

Table 1
Sociodemographic data of participants in the adaptation and validation stages.

Variables		Pre-test (n=30)	Validation (n=127)
		N (%)	N (%)
Sex	Female	20 (67)	89 (70.1)
	Male	10 (33)	37 (29.1)
	I’d rather not say		1 (0.8)
Education	First degree	09 (30)	45 (35.4)
	Diploma course	14 (47)	68 (53.5)
	Master’s degree	02 (07)	8 (6.3)
	Doctor’s degree	04 (13)	6 (4.7)
	Post-Doctoral degree	01 (03)	0
Main area of professional expertise	General Clinic	11 (37)	52 (40.9)
	Surgery	2 (7)	5 (3.9)
	Dentistry	1 (3)	6 (4.7)
	Endodontics	2 (7)	17 (13.4)
	Pediatric Dentistry	1 (3)	6 (4.7)
	Orthodontics	0	21 (16.5)
	Periodontics	1 (3)	11 (8.7)
	Lecturing	6 (20)	0
	Research	1 (3)	0
	Others	5 (17)	9 (7.2)
First degree obtained	Less than 1 year ago	03 (10)	9 (7.1)
	1 to 5 years ago	06 (20)	19 (15)
	5 to 10 years ago	0	11 (8.7)
	10 to 20 years ago	07 (23)	21 (16.5)
	More than 20 years ago	14 (47)	67 (52.8)
Have you ever provided dental treatment to any patient with diabetes?	Yes	30 (100)	120 (94.5)
	No	0	7 (5.5)
Have you ever taken any training or course about Diabetes?	Yes	10 (33)	10 (7.9)
	No	20 (67)	117 (92.1)
Do you feel empowered to provide care for Diabetes patients?	Yes	19 (63)	73 (57.5)
	No	11 (37)	54 (42.5)

As an outcome of the first meeting, with suggestions by 20 dentists, 2 questions were merged in order to adapt terms and increase understanding, a total of 22 items remaining in the questionnaire, yielding a third version (V3). V3 was tested with ten other dentists, no need for further redrafting having been requested. V3 was hence considered adequate to be submitted to psychometric validation. The 15 questions on knowledge about diabetes were then submitted for validation (Table 2).

Table 2
Items reviewed along the process of elaboration and adaptation.

	V1	V2	V3
Initial number of questions (Part 1/ Part 2)	23 (7/16)	23 (7/16)	22 (7/15)
Number of questions requiring redrafting or exclusion (Part 1/ Part 2)	21 (7/14)	9 (0/9)	0
Suggestions deemed necessary by the Expert Committee	17	5	NA
Questions excluded due to agreement below 80%	0	1	NA
Final number of questions (Part 1/ Part 2)	23 (7/16)	22 (7/15)	22 (7/15)

• V= version; Part 1: socio-demographic assessment; Part 2: knowledge assessment; NA: not applicable.

Validation

This stage lasted about 28 days and 127 dentists answered the final version of the instrument (test and retest) (Table 1). 70% were female, 53.5 % had a Diploma course, 40.9% had expertise in general clinical practice., and 52.8% had obtained their first degree 20 years ago. 94.5% had already provided dental care to people with diabetes.

The total CA alpha value was 0.794 (95% confidence interval 0.741- 0.842). Floor effects (percent with minimum score) were 0%, and ceiling effects (percent with maximum score) 3,2%. An ICC value of 0.799 (95% CI: 0.746-0.846) was obtained. The Kappa coefficient, which assesses the degree of agreement, varied between 0.5-1.0 (mean: 0.80). When the alpha absence index was calculated, there was a slight impact on reducing AC and no questions needed to be removed (Table 3).

Table 3
Correlation between test and retest, answers agreement percentage and Cronbach’s alpha coefficient for the Dental- Diabetes Instrument.

Item*	Kappa index	Percentual Agreement Test-Retest	Cronbach’s Alpha if item is removed	95% CI for alpha
Q1	0.629	76.27	0.813	0.763 – 0.857
Q2	0.368	61.34	0.787	0.729 – 0.837
Q3	0.658	79.17	0.797	0.743 – 0.845
Q4	0.594	88.50	0.798	0.745 – 0.846
Q5a	0.174	69.75	0.796	0.741 – 0.844
Q5b	0.22	85.12	0.796	0.741 – 0.844
Q5c	0.158	85.12	0.798	0.744 – 0.845
Q5d	0.501	97.58	0.797	0.743 – 0.845
Q5e	0.195	72.48	0.8	0.747 – 0.847
Q6	0.474	83.76	0.804	0.751 – 0.850
Q7a	0.335	91.13	0.794	0.794 – 0.842
Q7b	0.601	97.64	0.795	0.741 – 0.843
Q7c	0.315	76.72	0.792	0.736 – 0.840
Q7d	0.368	77.87	0.799	0.746 – 0.846
Q8	0.605	87.70	0.778	0.718 – 0.830
Q9	0.434	69.72	0.784	0.727 – 0.835
Q10	0.418	80.36	0.786	0.729 – 0.837
Q11	0.352	70.09	0.8	0.747 – 0.847
Q12a	0.478	79.65	0.8	0.746 – 0.847
Q12b	0.191	89.47	0.799	0.745 – 0.846
Q12c	0.239	85.84	0.796	0.741 – 0.844
Q13	0.675	89.34	0.798	0.744 – 0.845
Q14a	0.534	86.09	0.794	0.739 – 0.842
Q14b	0.391	80	0.793	0.738 – 0.842
Q14c	0.263	69.57	0.795	0.740 – 0.843
Q14d	0.3	7.27	0.796	0.741 – 0.844
Q15a	0.563	77.27	0.785	0.728 – 0.836
Q15b	0.464	76.85	0.787	0.730 – 0.837
Q15c	0.549	76.32	0.781	0.722 – 0.832
Q15d	0.464	84.35	0.781	0.723 – 0.833
Q15e	0.544	73.50	0.784	0.727 – 0.835

• * Instrument in Supplementary File.

The mean final score during test was 19.40, with a standard deviation of 4.49. At the time of the test, the percentages of minimum and maximum correct answers were 8.7% (question 2) and 95.3% (question 5d), respectively. All participants spent between 22 and 16 minutes on testing and retesting. The final version of the instrument is available in Supplementary Material.

Discussion

The treatment of patients with diabetes requires knowledgeable professionals, dentists being fundamental member in a multidisciplinary team; therefore, dentists are expected to be updated regarding diabetes and its implications for daily care, with a greater knowledge about the onset, duration and control of the disease, resulting in a more effective and satisfactory approach¹⁹. A good interaction between the dentist and the multidisciplinary team is essential for a safer dental treatment, with lower chances of complications for the patient²⁰.

Given the importance of the dental approach for the patient with DM, elaborating and validating an instrument to evaluate the dentists’ knowledge about DM was considered important to identify possible flaws in the knowledge of diabetes of these professionals that could impact treatment decisions and the objectives of the patient. In Brazil, dentists are not yet part of teams in diabetes centers as is the case in other countries, despite the clear need for these professionals to share the knowledge and duties of a multidisciplinary team.

The collaborative work by the Expert Committee pooling expertise in diabetes, dentistry and language issues made it possible to elaborate a comprehensive instrument, solving problems encountered during the process of drafting and adapting concepts and terms to the language used by the target subjects¹³,¹⁶,¹⁸,²¹-²³. Interaction between healthcare professionals and applied linguists is a fundamental piece in the elaboration and cultural adaptation of new instruments.

Assessment by the Expert committee through the web tool e-Surv is a reliable and efficient methodology²⁴, allowing for remote application and quick data extraction, avoiding potential errors in transcriptions and gathering of results²⁵. The instrument successfully passed the Committee’s examination in terms of clarity and relevance, with an excellent CVI (0.95). It should be noted that the maximum value for CVI is equal to 1, the results achieved being close to the maximum score¹⁸, well above the CVI cutoff point of 0.80 for new instruments¹⁸,²⁶.

In the pre-test, the face-to-face meetings with a sample of dentists proved successful, favoring adjustments in the instrument, and ensuring the prospective understanding of the items by the target audience²⁷,²⁸. In carefully developed instruments, two or three face-to-face tests can be satisfactory, which was the case in our study, with two rounds being required²¹.

For validation (test-retest), a CA index of 0.794 was obtained, which indicates good internal consistency²⁹-³². The time span for retest met the recommendations in the literature: a 7 to 21 day interval (a mean of 16 days). There are controversies regarding interval between test and retest, a desirable interval being not too short for participants to recall their answers in the test and too long for the study to be impacted³³-³⁵. There are recommendations of an interval of one to two weeks between test and retest; however, no fixed amount of time is prescribed, the main concern being the need to account for whatever interval span chosen³⁶. The time interval in our study adhered to the above recommendations, variation being due to participants’ agendas.

Our CA, ICC (0,799) and Kappa index (mean 0.80) indicate that our instrument showed adequate stability, reproducibility and confidence¹⁸. ICC being satisfactory, we computed Kappa to corroborate it. Items with a low Kappa (5a, 5c e 5d, 12b e 12c) revealed topics that were less familiar to dentists.

CA absence index was carried out. Removing items (1, 5e, 6, 11 e 12a) yielded alpha scores higher than those for the whole set of items. Therefore, no questions were excluded, due to the small difference that would result in the final CA³⁷ and the possibility of leaving out important information¹³. In addition, the value was above 0.799 for all items and thus considered satisfactory.

When we analyzed the performance of dentists in the test, the questions that had the lowest percentage of correct answers were question 2 (time period considered in the glycated hemoglobin test to assess mean blood glucose levels); 5 (hypoglycemia and signs of mood change and/or irritability); and 14c (use of sedatives). This percentage of incorrect answers was somehow predictable, since those are the most common questions asked to endocrinologists by dentists before dental procedures in daily care. In this respect, it is worthy of note that both in the pre-test and the validation stages, the majority of participants had over twenty years’ experience and had provided dental care to patients with diabetes; nevertheless, most had had no training whatsoever in diabetes care. Still, despite the lack of training, most reported feeling confident to treat patients with diabetes. This finding reveals a major problem in diabetes education and clinical practice regarding a highly prevalent condition as is diabetes and showcases the need for questionnaires such as the one we have elaborated and validated in our study as an instrument to assess the level of knowledge about diabetes by dentists and propose educational initiatives to contribute to better dental treatment for people with diabetes.

No studies were found describing the development and validation of instruments to assess dentists’ knowledge about diabetes, which did not allow for our results to be compared. Our instrument comprises items implicating knowledge compatible with themes indicated as priorities for dentists’ care of patients with diabetes: diagnostic criteria, symptoms, urgency and emergency, dental risk, conduct in care, clinical signs in the oral cavity and use of anesthetics and medications.

In conclusion, our study yielded an instrument that proved useful, reliable and stable for use by dentists. The instrument is useful to evaluate dentists’ knowledge and promote professionals’ training, with potential impact to enhance treatment for people with diabetes.

References

• ¹
  Saeedi P, Petersohn I, Salpea P, Malanda B, Karuranga S, Unwin N, et al. Global and regional diabetes prevalence estimates for 2019 and projections for 2030 and 2045: results from the International Diabetes Federation Diabetes Atlas, 9th edition. Diabetes Res Clin Pract. 2019 Nov;157:107843. doi: 10.1016/j.diabres.2019.107843.
  » https://doi.org/10.1016/j.diabres.2019.107843
• ²
  Brazilian Diabetes Society. [Guidelines of the Brazilian Diabetes Society 2019-2020]. São Paulo: Clannad; 2019 [cited 2021 Feb 22]. 289 p. Available from: https://www.diabetes.org.br/profissionais/images/DIRETRIZES-COMPLETA-2019-2020.pdf Portuguese.
  » https://www.diabetes.org.br/profissionais/images/DIRETRIZES-COMPLETA-2019-2020.pdf
• ³
  Indurkar MS, Maurya AS, Indurkar S. Oral manifestations of diabetes. Clin Diabetes. 2016 Jan;34(1):54-7. doi: 10.2337/diaclin.34.1.54.
  » https://doi.org/10.2337/diaclin.34.1.54
• ⁴
  Kudiyirickal MG, Pappachan JM. Diabetes mellitus and oral health. Endocrine. 2015 May;49(1):27-34. doi: 10.1007/s12020-014-0496-3.
  » https://doi.org/10.1007/s12020-014-0496-3
• ⁵
  Brazil, Ministry of Health of Brazil, Notebooks for Basic Care. [Strategies for care of the person with chronic disease: diabetes mellitus]. Brasília: Ministry of Health; 2014. p.138-41. Portuguese.
• ⁶
  Mauri-Obradors E, Estrugo-Devesa A, Jané-Salas E, Viñas M, López-López J. Oral manifestations of Diabetes Mellitus. A systematic review. Med Oral Patol Oral Cir Bucal. 2017 Sep;22(5):e586-e594. doi: 10.4317/medoral.21655.
  » https://doi.org/10.4317/medoral.21655
• ⁷
  Nazir MA, AlGhamdi L, AlKadi M, AlBeajan N, AlRashoudi L, AlHussan M. The burden of diabetes, its oral complications and their prevention and management. Open Access Maced J Med Sci. 2018 Aug;6(8):1545-53. doi: 10.3889/oamjms.2018.294.
  » https://doi.org/10.3889/oamjms.2018.294
• ⁸
  Brazil, Ministry of Health of Brazil. [Oral health]. (Series A. Standards and Technical Manuals - Basic Care Notebooks; 17). Brasília: Ministry of Health; 2008 [cited 2021 Feb 25]. 92p. Available from: https://bvsms.saude.gov.br/bvs/publicacoes/saude_bucal.pdf Portuguese.
  » https://bvsms.saude.gov.br/bvs/publicacoes/saude_bucal.pdf
• ⁹
  Beagley J, Guariguata L, Weil C, Motala AA. Global estimates of undiagnosed diabetes in adults. Diabetes Res Clin Pract. 2014 Feb;103(2):150-60. doi: 10.1016/j.diabres.2013.11.001.
  » https://doi.org/10.1016/j.diabres.2013.11.001
• ¹⁰
  Pilatti LA, Pedroso B, Gutierrez GL. [Psychometric Properties of Assessment Instruments: A Necessary Debate]. RBECT. 2010;3(1):81-91. Portuguese. doi: 10.3895/S1982-873X2010000100005.
  » https://doi.org/10.3895/S1982-873X2010000100005
• ¹¹
  Pasquali L. [Psychometrics]. Rev Esc Enferm USP. 2009;43(spe):992–9. Portuguese.
• ¹²
  Leite SS, Áfio ACE, Carvalho LV, Silva JM, Almeida PC, Pagliuca LMF. [Construction and validation of an Educational Content Validation Instrument in Health]. Reben. 2018;71(suppl 4):1635–41. Portuguese. doi: 10.1590/0034-7167-2017-0648.
  » https://doi.org/10.1590/0034-7167-2017-0648
• ¹³
  Fernandes BSM, Reis IA, Pagano AS, Cecilio SG, Torres HC. [Development, validation and cultural adaptation of the COMPASSO protocol: Adherence to self-care in diabetes]. Acta Paul Enferm. 2016;29(4):421–9. Portuguese. doi: 10.1590/1982-0194201600058.
  » https://doi.org/10.1590/1982-0194201600058
• ¹⁴
  Polit DF, Beck CT, Owen SV. Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Res Nurs Health. 2007 Aug;30(4):459-67. doi: 10.1002/nur.20199.
  » https://doi.org/10.1002/nur.20199
• ¹⁵
  Borsa JC, Damásio BF, Bandeira DR. [Cross-cultural adaptation and validation of psychological instruments: some considerations]. Paidéia (Ribeirão Preto). 2012 Dec;22(53):423–32. Portuguese. doi: 10.1590/S0103-863X2012000300014.
  » https://doi.org/10.1590/S0103-863X2012000300014
• ¹⁶
  Coluci MZO, Alexandre NMC, Milani D. [Construction of measurement instruments in the area of health]. Cien Saude Colet. 2015 Mar;20(3):925-36. Portuguese. doi: 10.1590/1413-81232015203.04332013.
  » https://doi.org/10.1590/1413-81232015203.04332013
• ¹⁷
  Souza AC, Alexandre NMC, Guirardello EB. Psychometric properties in instruments evaluation of reliability and validity. Epidemiol Serv Saude. 2017 Jul-Sep;26(3):649-659. doi: 10.5123/S1679-49742017000300022.
  » https://doi.org/10.5123/S1679-49742017000300022
• ¹⁸
  Alexandre NM, Coluci MZ. Validade de conteúdo nos processos de construção e adaptação de instrumentos de medidas [Content validity in the development and adaptation processes of measurement instruments]. Cien Saude Colet. 2011 Jul;16(7):3061-8. Portuguese. doi: 10.1590/s1413-81232011000800006.
  » https://doi.org/10.1590/s1413-81232011000800006
• ¹⁹
  Terra BG, Goulart RR, Bavaresco CS. [Dental care for patients with type 1 and 2 diabetes mellitus in primary health care]. Rev APS. 2011;14(2):149–61. Portuguese.
• ²⁰
  Tavares GR, Lima AL, Tavares SSS, Castro RD, Padilha WW. [Multiprofessional boarding to the carrying patient of diabetes in dentistry attendance]. Rev Flum Odontol. 2010; 16(33):52-59. Portuguese.
• ²¹
  Martins GA. [About reliability and validity]. RBGN. 2006;8(20):1-12. Portuguese.
• ²²
  Braga CG, da Cruz DA. Contribuições da psicometria para a avaliação de respostas psicossociais na enfermagem [Psychometric contributions to the assessment of psychosocial responses in nursing]. Rev Esc Enferm USP. 2006 Mar;40(1):98-104. Portuguese. doi: 10.1590/s0080-62342006000100014.
  » https://doi.org/10.1590/s0080-62342006000100014
• ²³
  Keszei AP, Novak M, Streiner DL. Introduction to health measurement scales. J Psychosom Res. 2010 Apr;68(4):319-23. doi: 10.1016/j.jpsychores.2010.01.006.
  » https://doi.org/10.1016/j.jpsychores.2010.01.006
• ²⁴
  Torres H de C, Chaves FF, Silva DDR da, Bosco AA, Gabriel BD, Reis IA, et al. [Translation, adaptation and validation the contents of the Diabetes Medical Management Plan for the Brazilian context]. Rev Latino-Am Enfermagem. 2016; 24:e2740. DOI: 10.1590/1518-8345.1138.2740. Portuguese.
  » https://doi.org/10.1590/1518-8345.1138.2740
• ²⁵
  Walter OM. [Analysis of free tools for conducting online surveys]. Produto & Produção. 2013;14(2):44-58. Portuguese. doi: 10.22456/1983-8026.22172.
  » https://doi.org/10.22456/1983-8026.22172
• ²⁶
  Medeiros RK, Ferreira Júnior M, Pinto D, Vitor A, Santos V, Barichello E. [Pasquali’s model of content validation in the Nursing researches]. Rev Enf Ref. 2015;4(4):123-31. Portuguese. doi: 10.12707/RIV14009.
  » https://doi.org/10.12707/RIV14009
• ²⁷
  Diniz KKS, Pagano AS, Fernandes APPC, Reis IA, Pinheiro Júnior LG, Torres HC. Development and validation of an instrument to assess Brazilian healthcare professional providers’ knowledge on sickle cell disease. Hematol Transfus Cell Ther. 2019 Apr-Jun;41(2):145-52. doi: 10.1016/j.htct.2018.08.003.
  » https://doi.org/10.1016/j.htct.2018.08.003
• ²⁸
  Silva TON, Alves LBO, Balieiro MMFG, Mandetta MA, Tanner A, Shields L. [Cross-cultural adaptation of an instrument to measure the family-centered care]. Acta Paul Enferm. 2015 april;28(2):107–12. Portuguese.
• ²⁹
  Maroco J, Garcia-Marques T. [How reliable is Cronbach’s alpha? Old issues and modern solutions?] Lab Psicol. 2006;4(1):65–90. Portuguese. doi: 10.14417/lp.763.
  » https://doi.org/10.14417/lp.763
• ³⁰
  Streiner DL. Being inconsistent about consistency: when coefficient alpha does and doesn’t matter. J Pers Assess. 2003 Jun;80(3):217-22. doi: 10.1207/S15327752JPA8003_01.
  » https://doi.org/10.1207/S15327752JPA8003_01
• ³¹
  Leal DL, Werneck MAF, Borges-Oliveira AC. [Validation of the oral health version of the Instrument for Diagnosis of the Developmental Stage of the Healthcare Network]. Rev Pan-Amaz Saude. 2017;8(4):65-75. doi: 10.5123/s2176-62232017000400011. Spanish.
  » https://doi.org/10.5123/s2176-62232017000400011
• ³²
  Vituri DW, Évora YDM. Reliability of indicators of nursing care quality: testing interexaminer agreement and reliability. Rev Latino-Am Enfermagem. 2014;22(2):234–40.
• ³³
  Frost MH, Reeve BB, Liepa AM, Stauffer JW, Hays RD. What is sufficient evidence for the reliability and validity of patient-reported outcome measures? Value Health. 2007;10 Suppl 2:S94-S105. doi: 10.1111/j.1524-4733.2007.00272.x.
  » https://doi.org/10.1111/j.1524-4733.2007.00272.x
• ³⁴
  Fayers P, Machin D. Quality of life: the assessment, analysis and interpretation of patient-reported outcomes. 2 ed. Chichester: John Wiley & Sons; 2007. 544 p.
• ³⁵
  DeVon HA, Block ME, Moyle-Wright P, Ernst DM, Hayden SJ, Lazzara DJ et al. A psychometric toolbox for testing validity and reliability. J Nurs Scholarsh. 2007;39(2):155-64. doi: 10.1111/j.1547-5069.2007.00161.x.
  » https://doi.org/10.1111/j.1547-5069.2007.00161.x
• ³⁶
  Terwee CB, Bot SD, de Boer MR, van der Windt DA, Knol DL, Dekker J et al. Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol. 2007 Jan;60(1):34-42. doi: 10.1016/j.jclinepi.2006.03.012.
  » https://doi.org/10.1016/j.jclinepi.2006.03.012
• ³⁷
  Valim MD, Marziale MH, Hayashida M, Rocha FL, Santos JL. Validity and reliability of the Questionnaire for Compliance with Standard Precaution. Rev Saude Publica. 2015;49:87. doi: 10.1590/S0034-8910.2015049005975.
  » https://doi.org/10.1590/S0034-8910.2015049005975

Edited by

Editor: Altair A. Del Bel Cury

Publication Dates

• Publication in this collection
  15 Apr 2022
• Date of issue
  2022

History

• Received
  19 Apr 2021
• Accepted
  27 Oct 2021