Dental Management Survey Brazil (DMS-BR): creation and validation of a management instrument

Abstract

Questionnaires for the assessment of knowledge and self-perception can be useful to diagnose what a dentist knows about management and administration. The aim of the present study was to create and validate the Dental Management Survey Brazil (DMS-BR) scale, based on meetings with experts in the field. After having elaborated the first version, 10 audits were performed in dental offices in order to produce the final version, which included nine dimensions: location, patient, finance, marketing, competition, quality, staff, career, and productivity. The accuracy of the instrument was measured by intrarater and interrater reliability. In the validation phase, 247 Brazilian dentists answered a web-based questionnaire. The data were processed using Stata 13.0 and the significance level was set at 95%. The instrument had intrarater and interrater reliability (ICC-0.93 and 0.94). The overall average of respondents for the DMS-BR scale was 3.77 (SD = 0.45). Skewness and kurtosis were below absolute values 3 and 7, respectively. Internal validity measured by Cronbach’s alpha was 0.925 and the correlation of each dimension with the final result of the DMS-BR ranged between 0.606 and 0.810. Correlation with the job satisfaction scale was 0.661. The SEM data ranged between 0.80 and 0.56. The questionnaire presented satisfactory indicators of dentists’ self-perception about management and administration activities.

Practice Management; Education; Dentistry

Introduction

The dental profession faces a number of current and future challenges. In some countries, such as the United States, the profession has a relatively good financial return and there is a shortage of professionals willing to engage in the academic career.^1,2 On the other hand, in other countries, such as Brazil and Japan, there is a surplus of dentists in the job market.^3,4

In both situations above, career possibilities for new dentists are changing because of both economic and dental practice management challenges.⁵ Healthcare professionals have to seek development in two professional activities: besides focusing on keeping their patients’ good oral health, they also have to organize management activities in their clinics.⁶

Some countries have begun to include management as a discipline in their undergraduate programs. In 1984, in the United States, they proposed a management curriculum, which has been updated many times, to be taught in dental schools.^{7,8,9,10,11,12} Other countries, and specifically Brazil, also propose the inclusion of management and administration topics in their curricula.¹³

Do new dentists have good knowledge of business administration, marketing, and management? How can we conduct a continuing professional development to enhance the knowledge acquired in college?¹⁴ Knowledge perception questionnaires can be useful to diagnose dentists’ knowledge about management and administration and to verify their strengths and weaknesses and, based on that, offer them continuing professional development.¹⁵

Accordingly, the aim of this study was to create and validate a Dental Management Survey Brazil (DMS-BR) scale to serve as a support tool for professionals and also to know their strengths and weaknesses in management in order to be able to update their knowledge continuously.

Method

Study design

A cross-sectional, descriptive, and analytical study was carried out for the development and validation of a questionnaire survey.

Development and validation of the DMS-BR

Three different steps were used for the development of the DMS-BR scale and its validation.

The proposed instrument was developed based on a literature review on management and marketing and its content validity was analyzed by a panel of professionals/researchers (Expert Panel) with expertise in marketing and management applied to dentistry.

Upon completion of the first version of the questionnaire, audits were carried out in a convenience sample of 10 dental offices in São Paulo, which had different characteristics (clinics and offices), with the objective of testing and reformulating the instrument if necessary.

During the interviews in this phase, the collected data were recorded in a form and then compared with the answers collected from the web-based questionnaire. The purpose was to check the consistency of what participants answered online compared to what was answered in the interview, when more in-depth questions were asked and the participants had the opportunity to explain their answers, demonstrating their knowledge and skills in management and marketing areas.

Upon completion of this step, in which all the necessary changes were made in the questions and after the questionnaire was considered easy to be understood and able to measure what had been proposed for the research, the second validation phase started. At this stage, the participants were asked to answer the web-based questionnaire twice within a 30-day interval.

Thereafter, the final validation step was initiated, which consisted in administering the questionnaire to Brazilian dentists. The final version of the instrument is shown in Tables 1 and 2. Participants were invited by e-mail, based on a list provided by the Regional Dental Association of São Paulo, with two invitations sent within approximately 30 days, and the questionnaire was promoted through the researcher’s profile on a social networking website. A total of 247 professionals were selected by convenience sampling and eventually participated in this phase.

Table 1
DMS-BR: dimensions 1 to 4.

Items	Dimension 1 - Location
01	How I classify the access to the clinic or office where I work;
34	Considering your target clientele, you evaluate the decor and comfort of your waiting room as:
37	I classify the condition and the appearance of the service room(s) as:
Items	Dimension 2 - Patient
02	Considering the relationship strategies with your patients such as calling old customers, sending direct mail, sending e-mail, among others, I consider my situation to be:
06	Considering the attention that I dedicate to my patient before, during and after the appointment, I classify my service as:
15	Regarding the observation and respect to appointment times, I consider my punctuality to be:
30	How I classify my (or my staff’s) ability to retain patients:
31	Considering ways to assess my clients’ satisfaction, I classify my situation as:
Items	Dimension 3 - Finance
03	Considering price research before a major purchase and financial reserve for the purchase of materials and equipment, I evaluate my behavior as:
23	I classify my current financial planning for future retirement as:
25	Considering that the price must be in accordance with your fixed and variable costs, how do you consider the definition of the prices charged in your office:
26	I classify the financial control that I have in my office or related to gains/professional expenses as:
38	How you rate your knowledge of the fixed and variable expenses related to your office:
Items	Dimension 4 - Marketing
04	Concerning the strategies to attract new customers to the office or clinic where I work, such as ads, partnerships, advertising on web search engines, among others, I consider my situation to be:
10	How I evaluate my knowledge in areas such as management, marketing and administration:
11	Considering the promotion of my services through printed or digital material, including social networking, website, brochure, ads, I consider my situation to be:
20	Considering my ability and dedication to the administrative part of the profession, I classify my work as:
35	I consider my knowledge of the profile of the patients I attend to, in other words, my target clientele, to be:

Table 2
DMS-BR: dimensions 5 to 9.

Items	Dimension 5 - Competition
07	I classify my competitive advantage over my main competitors as:
12	I classify my knowledge of my competitors as:
28	I classify my knowledge of the weaknesses and strengths of the office or clinic where I work as:
Items	Dimension 6 - Quality
05	Considering the quality and organization of data, I consider my patient records to be:
16	Considering the use of caps, masks, gloves, safety glasses and apron, I evaluate my personal protection as:
18	Analyzing the organization of my office, including files, records, materials and equipment, I classify my situation as:
21	Considering aspects such as storage, availability, loss and purchasing planning, I consider the materials inventory control of the place where I work to be:
24	Considering features that are important for me, I classify the performance of my main suppliers (e.g. dental and prosthesis laboratories) as:
29	As for the legal documentation, such as ISS, license from the Municipal Public Health Agency, etc. I classify the situation of the place where I work as:
Items	Dimension 7 - Staff
08	I would rate the quality of communication with my staff as:
13	Regarding the characteristics that I consider important, I classify the work of my employees and/or staff as:
16	I consider the attention given to comments or suggestions about the operation of the office made by my employees, my staff members or patients to be:
Items	Dimension 8 - Career
09	Regarding my professional life, I consider that planning strategies to achieve my future goals is:
17	Looking at the last four years, I consider the quantity and quality of the courses I attended to be:
19	Looking at the ergonomics of the service room, I consider the arrangement of furniture, equipment and materials to be:
27	Looking at the ergonomics of the service room, I consider the arrangement of furniture, equipment and materials to be:
Items	Dimension 9 - Productivity
14	Regarding the sales of my services, I consider my knowledge of negotiation skills and/or persuasive power over my customers to be:
22	I consider my productivity and work efficiency to be:
32	I consider the flow of patients (follow-up appointments, new appointments, missed appointments, late arrivals) where I work to be:
33	Given the amount of patients who seek me every month by word of mouth or upon the recommendation of colleagues, I consider it to be:

The final version of the instrument contains 38 questions covering nine dimensions: location, patient, finance, marketing, competition, quality, staff, career, and productivity. The questionnaire was structured using the Google Docs app (Google, USA), which includes a platform for the development of web-based questionnaires whereby one can create questions in various formats, such as checkboxes and multiple-choice questions. The data were stored using cloud computing, which consists in storing data on shared servers connected via the Internet and which can be exported to Microsoft Excel. The answers to the questionnaire were pre-established by the author as excellent, good, satisfactory, poor, and unsatisfactory. These responses were tabulated respectively as 5, 4, 3, 2, and 1. The average results obtained by the respondents in terms of knowledge and application of marketing and management tools was classified as follows: poor (1 to < 2), satisfactory/acceptable (> 2.0 to < 4.0), and good/excellent (> 4 to 5.0)

Data insertion and analysis

The data were collected on an answer sheet provided via Google Docs, stored using cloud computing, and exported into Microsoft Excel, where they were prepared for statistical analysis.

The statistical analysis was performed in Stata 13.0 and included frequency distribution, central tendency, measures of dispersion, minimum and maximum values, in addition to skewness (Sk) and kurtosis (Ku).

The participants’ answers on two different occasions were compared using Spearman’s correlation test.

Cronbach’s alpha was used to check the validity of the DMS-BR and Spearman’s correlation test was employed to verify the correlation between the DMS-BR overall score and the values of each dimension. Job satisfaction was assessed by the Dentist Satisfaction Survey (DSS)¹⁶ validated for Brazilian Portuguese.⁴ The results of DMS-BR and DSS scale were compared by Spearman’s correlation test. Structural equation modeling (SEM) was carried out with standardized coefficients. A 95% significance level was considered for all tests performed in this study.

Ethical considerations

The research project was approved by the Research Ethics Committee of the Dental School of University of São Paulo (protocol no. 206.342). In the phases carried out in the presence of the respondents, two copies of the Voluntary Informed Consent Form were filled out, as required by Resolution 196 of the National Committee for Research Ethics.

For the online stages, participants should read the Voluntary Informed Consent Form on the website homepage and accept to take part in the study, only then were they granted access to the questionnaire. At this stage, the researchers’ contact information was also made available so that the participants could contact them if they had any questions, ensuring the participants’ privacy at any time of the research.

Results

During scale creation, it was noted that the first version of the questionnaire did not meet the research objectives, some questions were difficult to understand, and the answers did not match reality, so the instrument was changed through the reformulation of questions and answers and a final version was produced (Tables 1 and 2).

Regarding the validation and reliability of the scale, results show that the intrarater and interrater reliability was high (intraclass correlation coefficients of 0.93 and 0.94).

The overall average of the respondents for the DMS-BR was 3.77 (SD = 0.45). The results of each dimension are presented in Table 3.

Table 3
DMS-BR: descriptive measures.

Dimensions	Average	Standard deviation	Min / Max	Sk/Ku
Location	4.23	0.54	2.33 / 5.00	-0.68 /3.54
Quality	4.10	0.47	2.33 / 5.00	0.17 / 2.58
Staff	4.02	0.56	1.00 / 5.00	-0.31 /3.10
Career	3.87	0.64	1.50 / 5.00	-0.58 /3.28
Patient	3.85	0.53	2.20 / 5.00	-0.37 /2.71
Productivity	3.73	0.64	1.75 / 5.00	-0.27 /2.68
Competition	3.68	0.62	1.67 / 5.00	-0.92 /3.92
Finance	3.47	0.72	1.20 / 5.00	-0.85 /3.94
Marketing	3.10	0.71	1.20 / 5.00	-0.68 /4.27
Total	3.77	0.44	2.24 / 4.84	-0.36 /3.10

• Min / Max: Minimum / maximum. Sk/Ku: Skewness and Kurtosis.

The Sk of the items was evaluated using summary statistics and was considered appropriate because all the Sk and Ku values were below absolute values 3 and 7, respectively (Table 3).

Internal validity measured by Cronbach’s alpha was 0.925. The correlation of each dimension with the final result of the DMS-BR ranged between 0.606 and 0.810. The correlation with the job satisfaction scale was 0.661 (Table 4).

Table 4
DMS-BR: Validity.

Dimensions	DMS-BR	Cronbach’s Alpha	DMS-BR/DSS
Location	0.810	0.598	0.661
Quality	0.799	0.632	0.541
Staff	0.792	0.792	0.523
Career	0.761	0.780	0.509
Patient	0.734	0.530	0.444
Productivity	0.705	0.653	0.432
Competition	0.705	0.666	0.431
Finance	0.699	0.780	0.415
Marketing	0.606	0.704	0.405
Total	1.000	0.925	0.661

• DMS-BR: correlation of each dimension with the final result of the DMS-BR/DSS: correlation with job satisfaction.

The SEM data ranged between 0.56 and 0.80. The likelihood ratio for x² (27) was 74.410 with a root mean squared error of approximation of 0.084 (Figure).

Figure
Structural equation modeling for DMS-BR.

Discussion

The sample consisted of dentists selected by convenience and invited to participate in the study by digital media. Compared to physical means such as regular mail, digital media bring a number of benefits and, therefore, they have gained momentum in recent years due to their practicality, conclusion of the process in less time and at lower costs, greater geographic coverage of population bases, and quality of response as satisfactory as that of physical means.¹⁷

The American Psychological Association states that descriptive measures of reliability and construct validity are critical aspects when evaluating psychometric scales. These components were sufficiently evaluated in this study.¹⁸

The best structure factor for the sample consisted of nine dimensions: location, patient, finance, marketing, competition, quality, staff, career, and productivity. The findings are consistent with the literature, indicating that the best design should include up to 10 items.¹⁹

The psychometric sensitivity of the items was evaluated using summary measures. The items were considered to have adequate psychometric sensitivity when Sk and Ku were below absolute values 3 and 7, respectively.²⁰

The reliability analysis yielded Cronbach’s alpha values of 0.925 and the variation among the dimensions fell between 0.598 and 0.792 (Table 4). These findings are within the acceptable values described in the literature.²¹

The DMS presented appropriate adjustment to the sample when the SEM and correlation values of the dimensions were used and the final results were also satisfactory.²⁰

The Dentist Satisfaction Survey (DSS)¹⁶ validated for Brazilian Portuguese⁴ was used to estimate the divergent validity of DMS-BR. There was a moderate association between DSS and DMS-BR, which is in accordance with the literature.⁴

Although there are no similar scales to the DMS-BR and since comparing the results in the literature is difficult, all the psychometric properties – validity, reliability and stability – were appropriate. Thus, the translation of DMS-BR into other languages and its validation are suggested as the subject of future studies.

Conclusion

The questionnaire provided good indicators of dentists’ self-perception about management and administration activities, fulfilling its purpose as a tool for professionals to know their strengths and weaknesses in management and to be able to update their knowledge continuously.

References

• ¹
  Haden NK, Weaver RG, Valachovic RW. Meeting the demand for future dental school faculty: trends, challenges, and responses. J Dent Educ. 2002;66(9):1102-13.
• ²
  Guthrie D, Valachovic RW, Brown LJ. The impact of new dental schools on the dental workforce through 2022. J Dent Educ. 2009;73(12):1353-60.
• ³
  Ryder MI, Morio, I. Current challenges for dental education in Japan and the United States. Jpn Dent Sci Rev. 2011;47(1):23-30. https://doi.org/10.1016/j.jdsr.2010.05.001
  » https://doi.org/10.1016/j.jdsr.2010.05.001
• ⁴
  Michel-Crosato E, Gonzales PS, Hebling E, Oliveira RN, Biazevic MG. Translation and validation of the Brazilian version of the Dentist Satisfaction Survey. Clin Lab Res Den. 2014;20(4):234-40. https://doi.org/10.11606/issn.2357-8041.clrd.2014.57061
  » https://doi.org/10.11606/issn.2357-8041.clrd.2014.57061
• ⁵
  Lange BM, Dunning DG, Qu W. An update on practice management instruction in U.S. Dental Schools. Dent Hypotheses. 2012;3(1):39-52. https://doi.org/10.4103/2155-8213.100495
  » https://doi.org/10.4103/2155-8213.100495
• ⁶
  Gonzales PS, Michel-Crosato E, Biazevic, MGH, Martins Filho IE. Web 2.0: How social networking sites are impacting Dentistry. Dent Open J. 2015;2(1)38-43. https://doi.org/10.17140/DOJ-2-108
  » https://doi.org/10.17140/DOJ-2-108
• ⁷
  Shard HA. American Association of Dental Schools, curricular guidelines for practice management. J Dent Educ. 1980;44(9):550-3.
• ⁸
  Abbott LJ, Farman G, Bauer LB. Practice management instruction in North American dental schools 1983-84. J Dent Educ. 1984;48(12):674-8.
• ⁹
  American Association of Dental Schools. Curriculum guidelines for practice administration. J Dent Educ. 1986;50(4):236-40.
• ¹⁰
  Ryder MI, Sargent P, Perry D. Evolution and revolution: the curriculum reform process at UCSF. J Dent Educ. 2008;72(12):1516-30.
• ¹¹
  Manski RJ, DiAngelis AJ, Williams JN, Cunningham, MA. Status of practice administration curricula. J Dent Educ. 1992;56(5):346-8.
• ¹²
  Lange BM, Dunning DG, Stewart DC, Hardage JL. The current state of practice administration curricula in U.S. dental schools. J Dent Educ. 1999;63(11):834-8.
• ¹³
  Zilbovicius C, Araujo ME, Botazzo C, Frias AC, Junqueira SR, Junqueira CR. A paradigm shift in predoctoral dental curricula in Brazil: evaluating the process of change. J Dent Educ. 2011;75(4):557-64.
• ¹⁴
  Hong SY. New frontiers in the continuum of dental education. N Y State Dent J. 2001;67(9):26-9.
• ¹⁵
  Streiner DL, Norma GR. Health measurement scales: a practical guide to their development and rise. 4th ed. New York: Oxford University; 2008.
• ¹⁶
  Jeong SH, Chung JK, Choi YH, Sohn W, Song KB. Factors related to job satisfaction among South Korean dentists. Community Dent Oral Epidemiol. 2006;34(6):460-6. https://doi.org/10.1111/j.1600-0528.2006.00297.x
  » https://doi.org/10.1111/j.1600-0528.2006.00297.x
• ¹⁷
  Kaplowitz MD, Hadlock TD, Levine R. A comparison of web and mail survey response rates. Public Opin Q. 2004;68(1):94-101. https://doi.org/10.1093/poq/nfh006
  » https://doi.org/10.1093/poq/nfh006
• ¹⁸
  Guyatt GH, Norman GR, Juniper EF, Griffith LE. A critical look at transition ratings. J Clin Epidemiol. 2002;55(9):900–8. https://doi.org/10.1016/S0895-4356(02)00435-3
  » https://doi.org/10.1016/S0895-4356(02)00435-3
• ¹⁹
  Lesage FX, Berjot S, Deschamps F. Psychometric properties of the French versions of the Perceived Stress Scale. Int J Occup Environ Health. 2012;25(2):178-84. https://doi.org/10.2478/s13382-012-0024-8
  » https://doi.org/10.2478/s13382-012-0024-8
• ²⁰
  Dias JCR, Maroco J, Campos JADB. Weight concerns scale applied to college students: comparison between pencil-and-paper and online formats. Cyberpsychol Behav Soc Netw. 2015;18(3):188-92. https://doi.org/10.1089/cyber.2014.0392
  » https://doi.org/10.1089/cyber.2014.0392
• ²¹
  Hair JF Jr, Black WC, Babin BJ, Anderson RE. Multivariate data analysis. 5th ed. Porto Alegre: Bookman; 2005.

Publication Dates

• Publication in this collection
  2017

History

• Received
  01 June 2016
• Reviewed
  13 Feb 2017
• Accepted
  03 Mar 2017