The use of tools to support oral lesion description in oral medicine referrals

Abstract:

Poor description of oral lesions jeopardize the prioritization of appointments in Oral Medicine. The present study investigated whether the use of support tools changes the quality of descriptions focusing on health care prioritization. Two oral lesions (A and B) were described by 64 dental students and 48 dentists using three methods: (a) without support tools (free); (b) using the oral examination form from the Specialties Manual in Oral Health/Brazilian Ministry of Health (SMOH form); and (c) using the OralDESC guideline. The descriptions were compared with a gold standard and percentage of agreement was analyzed by the Kruskal-Wallis and Dunn's tests (p<0.05). Descriptions with the OralDESC presented higher information quality. Considering items essential for prioritization, the OralDESC demonstrated better performance for lesion A; for lesion B, free descriptions and descriptions using the OralDESC were of higher quality than those using SMOH form. Therefore, the OralDESC offered greater support for the description of oral lesions for health care prioritization in Oral Medicine.

Keywords:
Primary Health Care; Secondary Care; Guideline; Oral Medicine; Referral and Consultation

Introduction

According to the guidelines of the Health Care Networks (RAS), the primary care at the Basic Health Units is the point of entry into the Brazilian Unified Health System (SUS).^1,2 Health issues that require specialized care are referred to the secondary and tertiary levels of care.² The flow of patients within the system, which must be coordinated by the primary care, is considered critical for the effectiveness of the SUS principles, in particular the principle of comprehensiveness.³ The consolidation of comprehensive care, one of the requirements of the National Oral Health Policy, depends on the relationship among the dental services in all levels of the health care system.⁴ Additionally, it is important that patients with more serious conditions have treatment priority to ensure equality in healthcare access.¹ Thus, to achieve comprehensiveness and continuity of care, healthcare professionals from different fields must work in multidisciplinary and interdisciplinary contexts.⁵

The referral of patients to other levels of care occurs for different reasons, such as the establishment or confirmation of a diagnosis, for treatment of a disease, for conducting specific tests (not available in primary care) or to obtain a second opinion on the management of a disease.⁶ Referral documents are important and quite common instruments for exchange of information between professionals from different health care levels.^7,8,9 In some situations, referral documents are the only means available for this exchange,^10,11,12 and may also affect the quality of care offered to the patients.^9,10,13 The quality of these documents is therefore essential for proper health care, and its contents should allow assessing the clinical need and the urgency for appointments.⁹

In order to be useful, referral documents must provide organized information, rhetorical relevance, appropriate level of detail, and be concise.^12,14 However, several studies have reported that referral letters often lack important information,^{7,8,9,10,12,13,15,16} which can result in problems such as improper scheduling, prioritization errors,^9,11,13 unnecessary repetition of tests⁷ and, ultimately, inadequate care and delay in treating the illness.¹⁰

Several studies aimed at defining the necessary information for the referral of patients.^{8,9,10,12,16,17} Depending on the specialty, the clinical description of the lesion is considered essential.^9,12,13,16 Lesion details may include type (primary morphology),^13,16 size,^9,16 location,^12,13,16 color and eventual ulceration,^9,13 shape and consistency,⁹ thickness, texture, presence of rolled margin (in the case of oral squamous cell carcinoma),¹³ duration,^12,16 symptoms,^12,16 clinical diagnosis,^12,16 suspicion of malignancy,¹⁷ risk factors for disease, and management details, such as use of medications/mouthwash, previous biopsy or denture/tooth adjustment.¹³

Specific strategies have been suggested to improve the content of referral documents, such as the use of support tools,^6,10,16 investment in education and training of professionals,^6,10 and the use of electronic referrals.^8,18,19 Some studies evaluated the quality of the referral documents specifically within the Oral Medicine field.^9,12,13,16 The use of guidelines or forms seems to result in the inclusion of a greater amount of information and detail in the referral documents, resulting in improved quality.^{9,10,13,16,20} Some authors, however, found no improvement with the use of these instruments.^8,21

The lack of detailed descriptions of oral lesions can lead to difficulties in prioritizing appointments in Oral Medicine.^9,12,13 In the metropolitan area of Florianopolis, state of Santa Catarina – Brazil, the referral to secondary and tertiary levels of care is carried out by the National Regulatory System (SISREG).²² In this system, the requesting professional fills an unstructured form with the description of the lesion. Referral documents with insufficient descriptions are pending in the system, requiring further information from the requesting professional,²³ which delays the appointment.

In order to improve the quality of the referral documents in the Oral Medicine services in the metropolitan area of Florianopolis, the OralDESC guideline was developed²⁴. The present study aimed to evaluate the effectiveness of OralDESC in increasing the quality of descriptions, and verify its potential for future use in clinical practice. The objectives thus were: 1) to compare the quality of descriptions of oral lesions using three different methods and 2) evaluate the potential of OralDESC to support the prioritization of the referral process in Oral Medicine.

Material and methods

This is a cross-sectional study, of descriptive and analytical nature. The study was approved by the Ethics Committee on Human Research of the Federal University of Santa Catarina (UFSC) (protocol number 2172 of October 17, 2011). The intentional sample was composed of students attending the last year of the Undergraduate School of Dentistry at UFSC and dentists from the primary health care services of Florianopolis and Biguaçu, Santa Catarina, Brazil.

Data collection took place in four different times, according to the availability of the participants. The students were divided into two groups (9th and 10th academic terms). Data collection from dentists occurred during Municipal Health Departments meetings.

Participants were asked to describe two oral lesions (Figure 1). To allow the participants to observe and describe the lesions under similar conditions, photographs of the lesions were projected on a white screen using multimedia equipment (Epson LCD projector, the EMP-S4 model, Epson America Inc., Long Beach, CA, USA). The device resolution was 800 × 600 pixels and images had approximately 170 × 110 cm dimension when projected on the screen.

Figure 1
Images of the lesions presented to the participants during the description process. Lesion A: leukoplakia. Lesion B: benign tumor of salivary gland. Source: Oral Pathology/UFSC collection.

Each lesion was described consecutively using three methods:

1. Method 1 (M1): free description, without assistance of any support instrument;

2. Method 2 (M2): the description was supported by the intra and extra oral physical examination form published in the Specialties Manual in Oral Health - Brazilian Ministry of Health (SMOH form);²⁵

3. Method 3 (M3): the description was supported by OralDESC guideline.²⁴

The potential knowledge accumulation due to successive presentation of the same lesion for the three methods was considered of little influence on the results, as repeated descriptions of the same lesion by the same person in a short period of time probably would not change significantly without the aid of support tools.

The SMOH form consists of a sequence of items: location, size, color, shape, base of attachment, consistency, mobility, secondary characteristics, etiology, affected lymph nodes, other lesions and clinical diagnosis. Each of these items is followed by a blank field for recording the description. OralDESC (Figure 2) is also organized in items, however instead of the blank fields it offers a list of descriptors or questions related to each main item. Additionally, images illustrate the descriptors related to lesion type, such as for base of attachment and surface features.

Figure 2
OralDESC guideline: items for oral lesion description and corresponding figures.

The data were collected in specific forms with blank fields to record full descriptions. Participants were also allowed to register any relevant question they would ask a patient with such a lesion. Five minutes were given for each description method. Therefore, the form for M1 was collected as soon as the participants finished filling it out or at the end of the 5 min, and a new form was provided for the second report. The same procedures were used for M2 (with the SMOH tool) and M3 (with the OralDESC).

For evaluation of the information quality, the descriptions were compared with a gold standard, established by three professors with PhD in Oral Medicine, with at least ten years of clinical experience. It was determined that the gold standards should include only the descriptive items essential to the referral process - including anamnesis questions, if applicable. The gold standard descriptions for lesions A and B are presented in Table 1.

Table 1
Gold standard descriptions for lesions A and B.

Items		Lesion A (Figure 1A)	Lesion B (Figure 1B)
Lesion Type (primary morphology)		Plaque	Tumor/nodule
Additional information		Cannot be rubbed off	Central ulcer approximately 0.5 cm diameter
Anatomical site / Location
	Structure affected	Tongue (border/ventral)	Soft/hard palate
	Side affected	Side (right)	Left side/from tooth 23 to 27
	Color	White	Similar to adjacent mucosa
	Surface features	Irregular surface	Smooth surface
	Size (approximate)	3.0 x 1.0 cm	5 cm in the larger diameter
Additional characteristics
	Consistency	Consistency	Consistency
	Base of attachment	-	Sessile base
	Growth rate	Growth rate	Growth rate
	Pain	Symptomatology	Symptomatology
Addiction / Risk factors
	Tobacco	Smoking history	-
	Alcohol	Alcohol abuse history	-

The descriptions from the participants were compared to the gold standard and the score of 1 was assigned for agreements. Synonyms of the gold standard were considered agreements if they provided the necessary information.

To evaluate the potential of OralDESC as a support tool for setting priorities, the comparisons considered the characteristics deemed essential to prioritization. Similar to the full description analysis, agreement scores were calculated. For lesion A, items considered essential were lesion type (plaque), color, (white), additional information (the lesion cannot be rubbed off), location in tongue (border/ventral), and risk factors (smoking history). For lesion B, the essential items were lesion type (tumor/nodule), affected site (palate), and questions about growth rate and pain.

For statistical analysis, data were described, and the Kruskal-Wallis test for independent samples and the Dunn's test were used assuming a 5% significance level. The null hypothesis was that there is no difference between free descriptions and descriptions supported by the SMOH form and the OralDESC guideline.

Results

The sample was composed by 64 students and 48 dentists (n = 112), totaling 336 descriptions for each lesion. Of the students, 31 were in the 9th academic term (80.6% female and 19.4% male) and 33 in the 10th academic term (81.8% female and 18.2% male). Of dentists, 70.8% were female and 20.8% male (8.4% did not inform), 25% had up to 5 years since dental school graduation, 22.9% had 6 to 10 years, 6.2% had 11 to 15 years, 12.5% had 16 to 20 years and 25% had over 21 years (8.4% did not report).

Considering information quality for lesion A, the average agreement rates for the free descriptions (M1), descriptions supported by the SMOH form (M2) and descriptions supported by the OralDESC guideline (M3) were 52.0%, 54.2% and 73.4%, respectively (Figure 3). According to the Kruskal-Wallis test results, the null hypothesis was rejected. In Table 2, the results of the between group analyses are shown. There was no difference between free descriptions and descriptions supported with the SMOH form (M1 and M2, respectively, p = 0.864). Using the OralDESC (M3) resulted in descriptions of higher quality for the lesion A, with a significant difference when compared to free descriptions (p < 0.05) and descriptions supported by the SMOH form (p < 0.05). No difference was observed between the descriptions from students and professionals among the three methods (p > 0.05).

Figure 3
Comparison between average agreement for lesions A and B.

Table 2
Analysis between groups (M1, M2 and M3) of lesions A and B descriptions, performed by the Dunn's test.

Groups		Statistical tests	Standard error	Standard test statistic	p-value
Lesion A
	M1 x M2	-13.670	12.867	-1.062	.864
	M1 x M3	-108.179	12.867	-8.408	.000
	M2 x M3	-94.509	12.867	-7.345	.000
Lesion B
	M1 x M2	-29.384	12.777	-2.300	.064
	M1 x M3	-87.375	12.777	-6.839	.000
	M2 x M3	-57.991	12.777	-4.539	.000

• *Asymptotic significances (two-tailed test) with a level of significance of 0.05.

For lesion B, the agreement average was 55.3% for free descriptions, 59.9% with the SMOH form and 70.8% with the OralDESC (Figure 3). A statistical difference was also observed among the three methods. The results from the between group analysis are shown in Table 2. No significant difference was observed between free descriptions and descriptions supported by the SMOH form (p = 0.064); however, the descriptions using the OralDESC were of significant higher quality compared to free descriptions (p < 0.05) and descriptions using the SMOH form (p < 0.05). Again, there was no difference between students and professionals responses (p > 0.05).

In the analysis of items considered essential for setting priorities, the average agreement for lesion A was 59.5% with free descriptions, 60% using the SMOH form and 73% using the OralDESC (Figure 4). A statistical difference was observed among the three methods. The between groups analysis (Table 3) revealed that descriptions with OralDESC were of higher quality than free descriptions (p<0.05) and the SMOH-supported reports (p < 0.05). There was no difference between free and SMOH-supported descriptions (p = 1.000).

Figure 4
Comparison between average agreement for items essential for setting priorities for lesions A and B.

Table 3
Analysis between groups (M1, M2 and M3) for the items considered essential for the process of setting priorities for lesions A and B, performed by the Dunn's test.

Groups		Statistical test	Standard error	Standard test statistic	p-value
Lesion A
	M2 x M1	1.857	12.488	.149	1.000
	M2 x M3	-58.411	12.488	-4.677	.000
	M1 x M3	-56.554	12.488	-4.529	.000
Lesion B
	M2 x M1	47.232	12.347	3.825	.000
	M2 x M3	-30.379	12.347	-2.461	.042
	M3 x M1	16.853	12.347	1.365	.517

• *Asymptotic significances (two-tailed test) with a level of significance of 0.05.

For lesion B, agreement for essential items for setting priorities was 80% for free descriptions, 66% for descriptions with the SMOH form and 75% for descriptions with the OralDESC (Figure 4). A statistical difference was also observed among the three methods. A significantly higher quality was observed for free descriptions compared to the SMOH-supported ones (p < 0.05). OralDESC provided better results than the SMOH form (p = 0.042), but similar to free descriptions (p = 0.517) (Table 3).

Discussion

The present study compared the information quality of oral lesion descriptions performed with and without support tools. The tools used were: 1) the SMOH form, designed to collect information on the admission of patients in secondary care units, and 2) OralDESC, designed specifically for the collection of information in primary care units aimed at prioritizing referrals. As no other support tool was available in the Brazilian literature, the SMOH form was used for comparative purposes with the OralDESC.

The results of the present study showed that despite the slight tendency for improvement in the quality of the descriptions using the SMOH form compared to free descriptions, it was not statistically significant for both lesions (p > 0.05). However, by using the OralDESC, a significant improvement was found compared to the other methods (p < 0.05). Thus, it seems reasonable to state that the use of support tools results in improved information quality, especially if the instrument is specifically designed for this purpose. Our results also seem to corroborate the initial estimate that knowledge accumulation due to the repetition of the procedure was unlikely, since there was no significant improvement from the first to the second time around (free descriptions and SMOH-supported descriptions).

The SMOH form addresses features commonly used for describing oral lesions, and its effective employment requires users to be familiar with the adequate semiological descriptors of a lesion to fill in the blanks. However, dentists do not commonly use this terminology in their daily clinical practice. By providing the descriptors, the OralDESC proved to be effective in improving the quality of oral lesions description, whether used by primary health care professionals (non-specialists) or by dental students. Moreover, the OralDESC provides images aimed at helping the users relate the terminology to specific clinical features,²⁴ which might also have played an important role in the results since visual elements appear to be useful in information processing.²⁶

Some relevant aspects observed during data analysis should be highlighted. One of them was the difficulty of some participants to properly describe the lesions, even concerning easily observable features such as lesion location. Perhaps such difficulties can be partially justified by the methodological limitation of using projected images of lesions. Another unexpected observation was the lack of familiarity with the correct application of semiologic terms for primary morphology of the lesions (also called lesion type), widely used by the medical and dental professionals. In some instances, this difficulty was evident even when the descriptors were coupled with figures, as in the case of the OralDESC guideline. Such difficulties may indicate a possible flaw in the training and/or lack of proper continuing education specific for Oral Medicine. Also, some studies report a lapse in the medical/dental curriculum with regard to written communication training.²⁷

Another point to be considered is the time required for preparation of referral documents with appropriate content; limited time has been appointed as a probable cause of referral letters with poor content.^13,17 In the present study, the time to perform each description was controlled, since it is possible that the clinical routine of Brazilian dentists in the primary health care does not permit sufficient time to perform proper referral documents to other levels of care.

Some authors reported that the years of experience influence the quality of referral documents with a tendency for better reports for professionals with less time since graduation.¹³ In the present study, there was no significant difference in the descriptions performed by students and dentists.

Jiwa et al.¹¹ argue that brief or incomplete referral documents may result in inappropriate scheduling of appointments in urgent cases. Foot, Naylor and Imison⁶ (2010) stressed the importance of properly reporting the information, especially if there is certainty or suspicion of a serious problem, as in cases of cancer. There are reports of delay in patient care as a result of inadequate referral documents in cases of oral cancer and its precursor lesions.^12,28 It is essential that professionals from the primary level of care provide information about risk factors in the referral letter such as smoking and alcohol consumption, known to be involved in the carcinogenesis of the most prevalent oral cancers. This information is requested in a clear and objective way in OralDESC.

Some participants provided a diagnostic hypothesis for the lesions - although this was not requested. Some authors have suggested that it is important to include a possible diagnosis in referral documents,¹⁰ as one of the effects of a better description is precisely to facilitate the clinical diagnosis. However, the appropriate description of the lesion also could help define priorities, considering characteristics like lesion profile, clinical management performed by the primary care professional, previous treatments, and additional information. A proper description may also allow the anticipation of complementary exams before the appointment with the specialist. On the other hand, an erroneous diagnostic hypothesis raised by the primary care professional could compromise the prioritization of the patient, delaying final diagnosis and hence the treatment.

In general, the descriptions supported by OralDESC showed greater standardization of the terminology. Additionally, the descriptions were more organized (following the topics proposed in the guideline), concise and contained more information. Appropriate, organized and concise information as well as standardized terminology potentially favor the elaboration of the differential diagnosis, facilitating prioritization. Some referral protocols in Oral Medicine are already available in Brazil, such as those used in the Brazilian National Telehealth Program of the state of Rio Grande do Sul.²⁹ Referral protocols, however, also depend on a good clinical description.

The standardization of terminology is an important point to be considered, particularly in cases of electronic referrals, which is becoming a trend with the implementation of electronic patient information systems. There are reports of successful experiences showing improved communication between professionals and decrease in waiting time for the patient,¹⁸ as well as faster inclusion and incorporation of patient data to the referral document.^17,19 In addition, electronic systems allow the inclusion of lesion images (or exams), which can increase the included information; however, the advantages of this aspect need to be better studied in future research.

It is important to note that most studies, including the present one, usually focus on a single aspect of the referral process. This is a limitation, since present deficiencies generally will not be solved with only one intervention.³⁰ Another limitation of the present study was the use of projected images of oral lesions, which restricts the collection of clinical information; to compensate this limitation a specific field for clinical description of the lesions was provided on the form.

Conclusions

Oral lesion descriptions using the OralDESC guideline demonstrated increased information quality when compared to free descriptions or those undertaken with support of the SMOH form. Similarly, considering items essential for prioritization, the OralDESC seems to have the potential to improve the prioritizing process in Oral Medicine when compared to free descriptions.

More studies should assess the need to offer specific training to professionals on establishing priorities based on OralDESC-supported referrals. The development and evaluation of referral protocols specific for Oral Medicine is also important. Finally, the potential integration of tools for oral lesion description into electronic referral systems should be studied, as well as its impact in comprehensive care and efficiency in the health care system, strengthening the role of primary care as the coordinating arm of Health Care Networks of the Brazilian Unified Health System.

Acknowledgment

The authors thank the partnership with the Municipal Health Secretariat of Florianopolis and Biguaçu, particularly Marynes Terezinha Reibnitz and Bruno Celio da Silva. We would also like to thank the dentists Nilse Terezinha Rohden and Monica Netto Mello who coordinated the process of setting priorities on dental care in the metropolitan area of Florianopolis, state of Santa Catarina.

References

• ¹
  Ministério da Saúde (BR). Secretaria Nacional de Assistência à Saúde. ABC do SUS: doutrinas e princípios. Brasília, DF: Ministério da Saúde; 1990.
• ²
  Mendes EV. As redes de atenção à saúde. Brasília, DF: Organização Pan-Americana da Saúde; 2011.
• ³
  Fratini JRG, Saupe R, Massaroli A. Referência e contra-referência: contribuição para a integralidade em saúde. Cienc Cuidado Saúde. 2008;7(1):65-72. https://doi.org/10.4025/cienccuidsaude.v7i1.4908
  » https://doi.org/10.4025/cienccuidsaude.v7i1.4908
• ⁴
  Ministério da Saúde (BR), Secretaria de Atenção à Saúde, Coordenação Nacional de Saúde Bucal. Diretrizes da Política Nacional de Saúde Bucal. Brasília, DF: Ministério da Saúde; 2004.
• ⁵
  Piterman L, Koritsas S. Part I. General practitioner-specialist relationship. Intern Med J. 2005;35(7):430–4. https://doi.org/10.1111/j.1445-5994.2005.00855.x
  » https://doi.org/10.1111/j.1445-5994.2005.00855.x
• ⁶
  Foot C, Naylor C, Imison C. The quality of GP diagnosis and referral: an inquiry into the quality of general practice in England. London, UK King's Fund; 2010 [cited 2015 May 28]. Available from: https://www.kingsfund.org.uk/sites/files/kf/Diagnosis%20and%20referral.pdf
  » https://www.kingsfund.org.uk/sites/files/kf/Diagnosis%20and%20referral.pdf
• ⁷
  Tattersall MH, Butow PN, Brown JE, Thompson JF. Improving doctors' letters. Med J Aust. 2002;177(3):516-20.
• ⁸
  Chetcuti K, Farrugia R, Cassar K. GP referral letters: time for a template? Malta Med J. 2009;21(2):26-9.
• ⁹
  Moloney J, Stassen LFA. An audit of the quality of referral letters received by the Department of Oral and Maxillofacial Surgery, Dublin Dental School and Hospital. J Ir Dent Assoc. 2010;56(5):221-3.
• ¹⁰
  Ibiyemi O, Ibiyemi T. Quality and contents of referral letters from peripheral health centers to the dental centre of a teaching hospital, southwestern Nigeria. Acta Odontol Scand. 2012 Mar;70(2):165-8. https://doi.org/10.3109/00016357.2011.600712
  » https://doi.org/10.3109/00016357.2011.600712
• ¹¹
  Jiwa M, Arnet H, Bulsara M, Ee HC, Harwood A. What is the importance of the referral letter in the patient journey? A pilot survey in Western Australia. Qual Prim Care. 2009;17(1):31-6.
• ¹²
  Navarro CM, Onofre MA, Sposto MR. Referral letters in oral medicine: an approach for the general dental practitioner. Int J Oral Maxillofac Surg. 2001;30(5):448-51. https://doi.org/10.1054/ijom.2001.0108
  » https://doi.org/10.1054/ijom.2001.0108
• ¹³
  White DA, Morris AJ, Burgess L, Hamburger J, Hamburger R. Facilitators and barriers to improving the quality of referrals for potential oral cancer. Br Dent J. 2004;197(9):537-40. https://doi.org/10.1038/sj.bdj.4811800
  » https://doi.org/10.1038/sj.bdj.4811800
• ¹⁴
  Lingard L, Hodges B, MacRae H, Freeman R. Expert and trainee determinations of rhetorical relevance in referral and consultation letters. Med Educ. 2004;38(2):168-76. https://doi.org/10.1111/j.1365-2923.2004.01745.x
  » https://doi.org/10.1111/j.1365-2923.2004.01745.x
• ¹⁵
  Ferreira JBB, Mishima MS, Santos JS, Forster AC, Ferraz CA. O complexo regulador da assistência à saúde na perspectiva de seus sujeitos operadores. Interface (Botucatu). 2010;14(33):345-58. https://doi.org/10.1590/S1414-32832010000200009
  » https://doi.org/10.1590/S1414-32832010000200009
• ¹⁶
  Navarro CM, Miranda IA, Onofre MA, Sposto MR. Referral letters in oral medicine: standard versus non-standard letters. Int J Oral Maxillofac Surg. 2002;31(5):537-43. https://doi.org/10.1054/ijom.2002.0277
  » https://doi.org/10.1054/ijom.2002.0277
• ¹⁷
  Shaw LJ, Berker DA. Strengths and weaknesses of electronic referral : comparison of data content and clinical value of electronic and paper referrals in dermatology. Br J Gen Pract. 2007;57(536):223-4.
• ¹⁸
  Kim Y, Chen AH, Keith E, Yee HF, Kushel MB. Not perfect, but better: primary care providers' experiences with electronic referrals in a safety net health system. J Gen Intern Med. 2009;24(5):6149. https://doi.org/10.1007/s11606-009-0955-3
  » https://doi.org/10.1007/s11606-009-0955-3
• ¹⁹
  Jiwa M, Dhaliwal S. Referral Writer: preliminary evidence for the value of comprehensive referral letters. Qual Prim Care. 2012;20(1):39-45.
• ²⁰
  Akbari A, Mayhew A, Al-Alawi MA, Grimshaw J, Winkens R, Glidewell E et al. Interventions to improve outpatient referrals from primary care to secondary care. Cochrane Database Syst Rev. 2008;8(4):CD005471. https://doi.org/10.1002/14651858.CD005471
  » https://doi.org/10.1002/14651858.CD005471
• ²¹
  O'Brien K, Wright J, Conboy F, Bagley L, Lewis D, Read M et al. The effect of orthodontic referral guidelines: a randomised controlled trial. Br Dent J. 2000;188(7):392-7.
• ²²
  Rohden NT, Costa AM, Silva MLB. Análise sobre a inserção da Especialidade de Estomatologia do Hospital Universitário no Sistema de Regulação de Florianópolis e seus Benefícios na Prática Odontológica. In: Pereira MF, Costa AM, Moritz GO, Bunn A, orgs Contribuições para a gestão do SUS. Florianópolis: Fundação Boiteux; 2013. p. 25-45. (Coleção Gestão da Saúde Pública, Vol. 13).
• ²³
  Bender AS, Molina L, Mello ALSF. Absenteísmo na atenção secundária e suas implicações na atenção básica. Espaço Saúde. 2010;11(2):56-65.
• ²⁴
  Meurer MI, Zimmermann C, Grando LJ. Proposta de um roteiro de apoio à descrição de lesões bucais como instrumentalização para a comunicação profissional. Rev ABENO. 2015;15(3):2-15.
• ²⁵
  Ministério da Saúde (BR), Secretaria de Atenção à Saúde, Departamento de Atenção Básica. Manual de especialidades em saúde bucal. Brasília, DF: Ministério da Saúde; 2008.
• ²⁶
  Li W. A single picture is worth a thousand words: the effects of images on online learning content. Dev Sci. 2001;14(2):181-9.
• ²⁷
  François J. Tool to assess the quality of consultation and referral request letters in family medicine. Can Fam Physician. 2011;57:574-5.
• ²⁸
  White P, Singleton A, Jones R. Copying referral letters to patients: the views of patients, patient representatives and doctors. Patient Educ Couns. 2004;55(1):94-8. https://doi.org/10.1016/j.pec.2003.08.003
  » https://doi.org/10.1016/j.pec.2003.08.003
• ²⁹
  TelessaúdeRS. Portal de Educação. Protocolo de regulação em estomatologia: lesões de tecidos moles. Porto Alegre: TelesaúdeRS; 2014 [cited year month day]. Available from: http://www.ufrgs.br/educatele/biblioteca-1/protocolos/protocolo-de-regulacao-em-estomatologia-lesoes-de-tecidos-moles/view
  » http://www.ufrgs.br/educatele/biblioteca-1/protocolos/protocolo-de-regulacao-em-estomatologia-lesoes-de-tecidos-moles/view
• ³⁰
  Mehrotra A, Forrest CB, Lin CY. Dropping the baton: specialty referrals in the United States. Milbank Q. 2011;89(1):39-68. https://doi.org/10.1111/j.1468-0009.2011.00619.x
  » https://doi.org/10.1111/j.1468-0009.2011.00619.x

Publication Dates

• Publication in this collection
  2017

History

• Received
  06 Mar 2017
• Reviewed
  11 Sept 2017
• Accepted
  28 Sept 2017