Dental Management Survey Brazil (DMS-BR): creation and validation of a management instrument

Paola Sampaio GONZALES Ismar Eduardo MARTINS FILHO Maria Gabriela Haye BIAZEVIC Paulo Roberto da SILVA Edgard MICHEL-CROSATO About the authors

Abstract

Questionnaires for the assessment of knowledge and self-perception can be useful to diagnose what a dentist knows about management and administration. The aim of the present study was to create and validate the Dental Management Survey Brazil (DMS-BR) scale, based on meetings with experts in the field. After having elaborated the first version, 10 audits were performed in dental offices in order to produce the final version, which included nine dimensions: location, patient, finance, marketing, competition, quality, staff, career, and productivity. The accuracy of the instrument was measured by intrarater and interrater reliability. In the validation phase, 247 Brazilian dentists answered a web-based questionnaire. The data were processed using Stata 13.0 and the significance level was set at 95%. The instrument had intrarater and interrater reliability (ICC-0.93 and 0.94). The overall average of respondents for the DMS-BR scale was 3.77 (SD = 0.45). Skewness and kurtosis were below absolute values 3 and 7, respectively. Internal validity measured by Cronbach’s alpha was 0.925 and the correlation of each dimension with the final result of the DMS-BR ranged between 0.606 and 0.810. Correlation with the job satisfaction scale was 0.661. The SEM data ranged between 0.80 and 0.56. The questionnaire presented satisfactory indicators of dentists’ self-perception about management and administration activities.

Practice Management; Education; Dentistry

Introduction

The dental profession faces a number of current and future challenges. In some countries, such as the United States, the profession has a relatively good financial return and there is a shortage of professionals willing to engage in the academic career.11. Haden NK, Weaver RG, Valachovic RW. Meeting the demand for future dental school faculty: trends, challenges, and responses. J Dent Educ. 2002;66(9):1102-13.,22. Guthrie D, Valachovic RW, Brown LJ. The impact of new dental schools on the dental workforce through 2022. J Dent Educ. 2009;73(12):1353-60. On the other hand, in other countries, such as Brazil and Japan, there is a surplus of dentists in the job market.33. Ryder MI, Morio, I. Current challenges for dental education in Japan and the United States. Jpn Dent Sci Rev. 2011;47(1):23-30. https://doi.org/10.1016/j.jdsr.2010.05.001
https://doi.org/10.1016/j.jdsr.2010.05.0...
,44. Michel-Crosato E, Gonzales PS, Hebling E, Oliveira RN, Biazevic MG. Translation and validation of the Brazilian version of the Dentist Satisfaction Survey. Clin Lab Res Den. 2014;20(4):234-40. https://doi.org/10.11606/issn.2357-8041.clrd.2014.57061
https://doi.org/10.11606/issn.2357-8041....

In both situations above, career possibilities for new dentists are changing because of both economic and dental practice management challenges.55. Lange BM, Dunning DG, Qu W. An update on practice management instruction in U.S. Dental Schools. Dent Hypotheses. 2012;3(1):39-52. https://doi.org/10.4103/2155-8213.100495
https://doi.org/10.4103/2155-8213.100495...
Healthcare professionals have to seek development in two professional activities: besides focusing on keeping their patients’ good oral health, they also have to organize management activities in their clinics.66. Gonzales PS, Michel-Crosato E, Biazevic, MGH, Martins Filho IE. Web 2.0: How social networking sites are impacting Dentistry. Dent Open J. 2015;2(1)38-43. https://doi.org/10.17140/DOJ-2-108
https://doi.org/10.17140/DOJ-2-108...

Some countries have begun to include management as a discipline in their undergraduate programs. In 1984, in the United States, they proposed a management curriculum, which has been updated many times, to be taught in dental schools.77. Shard HA. American Association of Dental Schools, curricular guidelines for practice management. J Dent Educ. 1980;44(9):550-3.,88. Abbott LJ, Farman G, Bauer LB. Practice management instruction in North American dental schools 1983-84. J Dent Educ. 1984;48(12):674-8.,99. American Association of Dental Schools. Curriculum guidelines for practice administration. J Dent Educ. 1986;50(4):236-40.,1010. Ryder MI, Sargent P, Perry D. Evolution and revolution: the curriculum reform process at UCSF. J Dent Educ. 2008;72(12):1516-30.,1111. Manski RJ, DiAngelis AJ, Williams JN, Cunningham, MA. Status of practice administration curricula. J Dent Educ. 1992;56(5):346-8.,1212. Lange BM, Dunning DG, Stewart DC, Hardage JL. The current state of practice administration curricula in U.S. dental schools. J Dent Educ. 1999;63(11):834-8. Other countries, and specifically Brazil, also propose the inclusion of management and administration topics in their curricula.1313. Zilbovicius C, Araujo ME, Botazzo C, Frias AC, Junqueira SR, Junqueira CR. A paradigm shift in predoctoral dental curricula in Brazil: evaluating the process of change. J Dent Educ. 2011;75(4):557-64.

Do new dentists have good knowledge of business administration, marketing, and management? How can we conduct a continuing professional development to enhance the knowledge acquired in college?1414. Hong SY. New frontiers in the continuum of dental education. N Y State Dent J. 2001;67(9):26-9. Knowledge perception questionnaires can be useful to diagnose dentists’ knowledge about management and administration and to verify their strengths and weaknesses and, based on that, offer them continuing professional development.1515. Streiner DL, Norma GR. Health measurement scales: a practical guide to their development and rise. 4th ed. New York: Oxford University; 2008.

Accordingly, the aim of this study was to create and validate a Dental Management Survey Brazil (DMS-BR) scale to serve as a support tool for professionals and also to know their strengths and weaknesses in management in order to be able to update their knowledge continuously.

Method

Study design

A cross-sectional, descriptive, and analytical study was carried out for the development and validation of a questionnaire survey.

Development and validation of the DMS-BR

Three different steps were used for the development of the DMS-BR scale and its validation.

The proposed instrument was developed based on a literature review on management and marketing and its content validity was analyzed by a panel of professionals/researchers (Expert Panel) with expertise in marketing and management applied to dentistry.

Upon completion of the first version of the questionnaire, audits were carried out in a convenience sample of 10 dental offices in São Paulo, which had different characteristics (clinics and offices), with the objective of testing and reformulating the instrument if necessary.

During the interviews in this phase, the collected data were recorded in a form and then compared with the answers collected from the web-based questionnaire. The purpose was to check the consistency of what participants answered online compared to what was answered in the interview, when more in-depth questions were asked and the participants had the opportunity to explain their answers, demonstrating their knowledge and skills in management and marketing areas.

Upon completion of this step, in which all the necessary changes were made in the questions and after the questionnaire was considered easy to be understood and able to measure what had been proposed for the research, the second validation phase started. At this stage, the participants were asked to answer the web-based questionnaire twice within a 30-day interval.

Thereafter, the final validation step was initiated, which consisted in administering the questionnaire to Brazilian dentists. The final version of the instrument is shown in Tables 1 and 2. Participants were invited by e-mail, based on a list provided by the Regional Dental Association of São Paulo, with two invitations sent within approximately 30 days, and the questionnaire was promoted through the researcher’s profile on a social networking website. A total of 247 professionals were selected by convenience sampling and eventually participated in this phase.

Table 1
DMS-BR: dimensions 1 to 4.

Table 2
DMS-BR: dimensions 5 to 9.

The final version of the instrument contains 38 questions covering nine dimensions: location, patient, finance, marketing, competition, quality, staff, career, and productivity. The questionnaire was structured using the Google Docs app (Google, USA), which includes a platform for the development of web-based questionnaires whereby one can create questions in various formats, such as checkboxes and multiple-choice questions. The data were stored using cloud computing, which consists in storing data on shared servers connected via the Internet and which can be exported to Microsoft Excel. The answers to the questionnaire were pre-established by the author as excellent, good, satisfactory, poor, and unsatisfactory. These responses were tabulated respectively as 5, 4, 3, 2, and 1. The average results obtained by the respondents in terms of knowledge and application of marketing and management tools was classified as follows: poor (1 to < 2), satisfactory/acceptable (> 2.0 to < 4.0), and good/excellent (> 4 to 5.0)

Data insertion and analysis

The data were collected on an answer sheet provided via Google Docs, stored using cloud computing, and exported into Microsoft Excel, where they were prepared for statistical analysis.

The statistical analysis was performed in Stata 13.0 and included frequency distribution, central tendency, measures of dispersion, minimum and maximum values, in addition to skewness (Sk) and kurtosis (Ku).

The participants’ answers on two different occasions were compared using Spearman’s correlation test.

Cronbach’s alpha was used to check the validity of the DMS-BR and Spearman’s correlation test was employed to verify the correlation between the DMS-BR overall score and the values of each dimension. Job satisfaction was assessed by the Dentist Satisfaction Survey (DSS)1616. Jeong SH, Chung JK, Choi YH, Sohn W, Song KB. Factors related to job satisfaction among South Korean dentists. Community Dent Oral Epidemiol. 2006;34(6):460-6. https://doi.org/10.1111/j.1600-0528.2006.00297.x
https://doi.org/10.1111/j.1600-0528.2006...
validated for Brazilian Portuguese.44. Michel-Crosato E, Gonzales PS, Hebling E, Oliveira RN, Biazevic MG. Translation and validation of the Brazilian version of the Dentist Satisfaction Survey. Clin Lab Res Den. 2014;20(4):234-40. https://doi.org/10.11606/issn.2357-8041.clrd.2014.57061
https://doi.org/10.11606/issn.2357-8041....
The results of DMS-BR and DSS scale were compared by Spearman’s correlation test. Structural equation modeling (SEM) was carried out with standardized coefficients. A 95% significance level was considered for all tests performed in this study.

Ethical considerations

The research project was approved by the Research Ethics Committee of the Dental School of University of São Paulo (protocol no. 206.342). In the phases carried out in the presence of the respondents, two copies of the Voluntary Informed Consent Form were filled out, as required by Resolution 196 of the National Committee for Research Ethics.

For the online stages, participants should read the Voluntary Informed Consent Form on the website homepage and accept to take part in the study, only then were they granted access to the questionnaire. At this stage, the researchers’ contact information was also made available so that the participants could contact them if they had any questions, ensuring the participants’ privacy at any time of the research.

Results

During scale creation, it was noted that the first version of the questionnaire did not meet the research objectives, some questions were difficult to understand, and the answers did not match reality, so the instrument was changed through the reformulation of questions and answers and a final version was produced (Tables 1 and 2).

Regarding the validation and reliability of the scale, results show that the intrarater and interrater reliability was high (intraclass correlation coefficients of 0.93 and 0.94).

The overall average of the respondents for the DMS-BR was 3.77 (SD = 0.45). The results of each dimension are presented in Table 3.

Table 3
DMS-BR: descriptive measures.

The Sk of the items was evaluated using summary statistics and was considered appropriate because all the Sk and Ku values were below absolute values 3 and 7, respectively (Table 3).

Internal validity measured by Cronbach’s alpha was 0.925. The correlation of each dimension with the final result of the DMS-BR ranged between 0.606 and 0.810. The correlation with the job satisfaction scale was 0.661 (Table 4).

Table 4
DMS-BR: Validity.

The SEM data ranged between 0.56 and 0.80. The likelihood ratio for x2 (27) was 74.410 with a root mean squared error of approximation of 0.084 (Figure).

Figure
Structural equation modeling for DMS-BR.

Discussion

The sample consisted of dentists selected by convenience and invited to participate in the study by digital media. Compared to physical means such as regular mail, digital media bring a number of benefits and, therefore, they have gained momentum in recent years due to their practicality, conclusion of the process in less time and at lower costs, greater geographic coverage of population bases, and quality of response as satisfactory as that of physical means.1717. Kaplowitz MD, Hadlock TD, Levine R. A comparison of web and mail survey response rates. Public Opin Q. 2004;68(1):94-101. https://doi.org/10.1093/poq/nfh006
https://doi.org/10.1093/poq/nfh006...

The American Psychological Association states that descriptive measures of reliability and construct validity are critical aspects when evaluating psychometric scales. These components were sufficiently evaluated in this study.1818. Guyatt GH, Norman GR, Juniper EF, Griffith LE. A critical look at transition ratings. J Clin Epidemiol. 2002;55(9):900–8. https://doi.org/10.1016/S0895-4356(02)00435-3
https://doi.org/10.1016/S0895-4356(02)00...

The best structure factor for the sample consisted of nine dimensions: location, patient, finance, marketing, competition, quality, staff, career, and productivity. The findings are consistent with the literature, indicating that the best design should include up to 10 items.1919. Lesage FX, Berjot S, Deschamps F. Psychometric properties of the French versions of the Perceived Stress Scale. Int J Occup Environ Health. 2012;25(2):178-84. https://doi.org/10.2478/s13382-012-0024-8
https://doi.org/10.2478/s13382-012-0024-...

The psychometric sensitivity of the items was evaluated using summary measures. The items were considered to have adequate psychometric sensitivity when Sk and Ku were below absolute values 3 and 7, respectively.2020. Dias JCR, Maroco J, Campos JADB. Weight concerns scale applied to college students: comparison between pencil-and-paper and online formats. Cyberpsychol Behav Soc Netw. 2015;18(3):188-92. https://doi.org/10.1089/cyber.2014.0392
https://doi.org/10.1089/cyber.2014.0392...

The reliability analysis yielded Cronbach’s alpha values of 0.925 and the variation among the dimensions fell between 0.598 and 0.792 (Table 4). These findings are within the acceptable values described in the literature.2121. Hair JF Jr, Black WC, Babin BJ, Anderson RE. Multivariate data analysis. 5th ed. Porto Alegre: Bookman; 2005.

The DMS presented appropriate adjustment to the sample when the SEM and correlation values of the dimensions were used and the final results were also satisfactory.2020. Dias JCR, Maroco J, Campos JADB. Weight concerns scale applied to college students: comparison between pencil-and-paper and online formats. Cyberpsychol Behav Soc Netw. 2015;18(3):188-92. https://doi.org/10.1089/cyber.2014.0392
https://doi.org/10.1089/cyber.2014.0392...

The Dentist Satisfaction Survey (DSS)1616. Jeong SH, Chung JK, Choi YH, Sohn W, Song KB. Factors related to job satisfaction among South Korean dentists. Community Dent Oral Epidemiol. 2006;34(6):460-6. https://doi.org/10.1111/j.1600-0528.2006.00297.x
https://doi.org/10.1111/j.1600-0528.2006...
validated for Brazilian Portuguese44. Michel-Crosato E, Gonzales PS, Hebling E, Oliveira RN, Biazevic MG. Translation and validation of the Brazilian version of the Dentist Satisfaction Survey. Clin Lab Res Den. 2014;20(4):234-40. https://doi.org/10.11606/issn.2357-8041.clrd.2014.57061
https://doi.org/10.11606/issn.2357-8041....
was used to estimate the divergent validity of DMS-BR. There was a moderate association between DSS and DMS-BR, which is in accordance with the literature.44. Michel-Crosato E, Gonzales PS, Hebling E, Oliveira RN, Biazevic MG. Translation and validation of the Brazilian version of the Dentist Satisfaction Survey. Clin Lab Res Den. 2014;20(4):234-40. https://doi.org/10.11606/issn.2357-8041.clrd.2014.57061
https://doi.org/10.11606/issn.2357-8041....

Although there are no similar scales to the DMS-BR and since comparing the results in the literature is difficult, all the psychometric properties – validity, reliability and stability – were appropriate. Thus, the translation of DMS-BR into other languages and its validation are suggested as the subject of future studies.

Conclusion

The questionnaire provided good indicators of dentists’ self-perception about management and administration activities, fulfilling its purpose as a tool for professionals to know their strengths and weaknesses in management and to be able to update their knowledge continuously.

References

  • 1
    Haden NK, Weaver RG, Valachovic RW. Meeting the demand for future dental school faculty: trends, challenges, and responses. J Dent Educ. 2002;66(9):1102-13.
  • 2
    Guthrie D, Valachovic RW, Brown LJ. The impact of new dental schools on the dental workforce through 2022. J Dent Educ. 2009;73(12):1353-60.
  • 3
    Ryder MI, Morio, I. Current challenges for dental education in Japan and the United States. Jpn Dent Sci Rev. 2011;47(1):23-30. https://doi.org/10.1016/j.jdsr.2010.05.001
    » https://doi.org/10.1016/j.jdsr.2010.05.001
  • 4
    Michel-Crosato E, Gonzales PS, Hebling E, Oliveira RN, Biazevic MG. Translation and validation of the Brazilian version of the Dentist Satisfaction Survey. Clin Lab Res Den. 2014;20(4):234-40. https://doi.org/10.11606/issn.2357-8041.clrd.2014.57061
    » https://doi.org/10.11606/issn.2357-8041.clrd.2014.57061
  • 5
    Lange BM, Dunning DG, Qu W. An update on practice management instruction in U.S. Dental Schools. Dent Hypotheses. 2012;3(1):39-52. https://doi.org/10.4103/2155-8213.100495
    » https://doi.org/10.4103/2155-8213.100495
  • 6
    Gonzales PS, Michel-Crosato E, Biazevic, MGH, Martins Filho IE. Web 2.0: How social networking sites are impacting Dentistry. Dent Open J. 2015;2(1)38-43. https://doi.org/10.17140/DOJ-2-108
    » https://doi.org/10.17140/DOJ-2-108
  • 7
    Shard HA. American Association of Dental Schools, curricular guidelines for practice management. J Dent Educ. 1980;44(9):550-3.
  • 8
    Abbott LJ, Farman G, Bauer LB. Practice management instruction in North American dental schools 1983-84. J Dent Educ. 1984;48(12):674-8.
  • 9
    American Association of Dental Schools. Curriculum guidelines for practice administration. J Dent Educ. 1986;50(4):236-40.
  • 10
    Ryder MI, Sargent P, Perry D. Evolution and revolution: the curriculum reform process at UCSF. J Dent Educ. 2008;72(12):1516-30.
  • 11
    Manski RJ, DiAngelis AJ, Williams JN, Cunningham, MA. Status of practice administration curricula. J Dent Educ. 1992;56(5):346-8.
  • 12
    Lange BM, Dunning DG, Stewart DC, Hardage JL. The current state of practice administration curricula in U.S. dental schools. J Dent Educ. 1999;63(11):834-8.
  • 13
    Zilbovicius C, Araujo ME, Botazzo C, Frias AC, Junqueira SR, Junqueira CR. A paradigm shift in predoctoral dental curricula in Brazil: evaluating the process of change. J Dent Educ. 2011;75(4):557-64.
  • 14
    Hong SY. New frontiers in the continuum of dental education. N Y State Dent J. 2001;67(9):26-9.
  • 15
    Streiner DL, Norma GR. Health measurement scales: a practical guide to their development and rise. 4th ed. New York: Oxford University; 2008.
  • 16
    Jeong SH, Chung JK, Choi YH, Sohn W, Song KB. Factors related to job satisfaction among South Korean dentists. Community Dent Oral Epidemiol. 2006;34(6):460-6. https://doi.org/10.1111/j.1600-0528.2006.00297.x
    » https://doi.org/10.1111/j.1600-0528.2006.00297.x
  • 17
    Kaplowitz MD, Hadlock TD, Levine R. A comparison of web and mail survey response rates. Public Opin Q. 2004;68(1):94-101. https://doi.org/10.1093/poq/nfh006
    » https://doi.org/10.1093/poq/nfh006
  • 18
    Guyatt GH, Norman GR, Juniper EF, Griffith LE. A critical look at transition ratings. J Clin Epidemiol. 2002;55(9):900–8. https://doi.org/10.1016/S0895-4356(02)00435-3
    » https://doi.org/10.1016/S0895-4356(02)00435-3
  • 19
    Lesage FX, Berjot S, Deschamps F. Psychometric properties of the French versions of the Perceived Stress Scale. Int J Occup Environ Health. 2012;25(2):178-84. https://doi.org/10.2478/s13382-012-0024-8
    » https://doi.org/10.2478/s13382-012-0024-8
  • 20
    Dias JCR, Maroco J, Campos JADB. Weight concerns scale applied to college students: comparison between pencil-and-paper and online formats. Cyberpsychol Behav Soc Netw. 2015;18(3):188-92. https://doi.org/10.1089/cyber.2014.0392
    » https://doi.org/10.1089/cyber.2014.0392
  • 21
    Hair JF Jr, Black WC, Babin BJ, Anderson RE. Multivariate data analysis. 5th ed. Porto Alegre: Bookman; 2005.

Publication Dates

  • Publication in this collection
    2017

History

  • Received
    01 June 2016
  • Reviewed
    13 Feb 2017
  • Accepted
    03 Mar 2017
Sociedade Brasileira de Pesquisa Odontológica - SBPqO Av. Prof. Lineu Prestes, 2227, 05508-000 São Paulo SP - Brazil, Tel. (55 11) 3044-2393/(55 11) 9-7557-1244 - São Paulo - SP - Brazil
E-mail: office.bor@ingroup.srv.br