Psychometric analysis and reliability of the dental treatment motivation scale for Indonesian pregnant women

Abstract:

This study aimed to obtain the results of psychometric analysis and reliability of the dental treatment motivation scale (DTMS), which was adapted cross-culturally for pregnant women in Indonesia. A descriptive survey was conducted with 149 pregnant women at a maternal clinic in Bandung City, Indonesia, in December 2020. Convenience sampling was chosen as the sampling technique after the inclusion and exclusion criteria had been fulfilled. Two English experts initially translated the DTMS questionnaire from English into Indonesian and the translated questionnaire was then reviewed by four expert panels, which were modified afterwards. The validity test was carried out using the exploratory and confirmatory factor analyses and goodness of fit index (GFI). The internal reliability analysis used Cronbach’s alpha, and the test-retest was conducted using Pearson’s correlation coefficient. The psychometric analysis results were obtained from four testing stages. The first stage was the initial eigenvalue assessment, and the cumulative percentage value was >60%. The second stage was the exploratory factor analysis, with a loading factor of >0.3. The third stage consisted of the confirmatory factor analysis, forming two components of the factor structure (intrinsic and extrinsic motivations). Finally, the fourth stage was the GFI assessment, which showed the good fit model with a value of 0.903. Very high internal consistency reliability ranged from 0.985 to 0.990; the test-retest p-value of Pearson’s correlation coefficient was 0.000. In conclusion, the Indonesian version of the DTMS proved to be a reliable and valid instrument to measure Indonesian pregnant women’s motivation to seek oral health treatment.

Keywords:
Motivation; Oral Health; Psychometrics; Pregnant Women

Introduction

A questionnaire is one of the most widely used tools for collecting data, especially in social science research.¹ Most of the scientific questionnaires used in dentistry have been developed in English-speaking countries. However, many international multicentre studies, including populations with different cultural backgrounds and languages, are developed.² A questionnaire is not always correctly translated before being used in research and it is usually not adapted to a new culture or language. Therefore, the translation results may not accurately reflect the measurement.³

In addition, the translation process can create bias and affect cultural equality.⁴ Many researchers have demonstrated that multiple or panel or committee translations would provide better results than single translation, and it would be better to add psychometric measurements to this cross-cultural process.⁴,⁵ The process of cross-cultural adaptation seeks to produce equivalence between source and target based on content.² There is no universal agreement on how to adapt the instrument for use in other cultural environments³; however, WHO has issued a process of adaptation and translation of instruments to guide cross-cultural implementation.⁶

The primary purpose of a research questionnaire is to obtain relevant information. Therefore, accuracy and consistency are essential aspects of the research methodology, known as validity and reliability.¹ Validity in research refers to how accurately a study answers the research question or generates conclusions.⁷-⁸,⁹ Reliability refers to the ability of an assessment instrument to provide the same results whenever used in the same setting with the same subject type. Thus, reliability determines consistent or reliable results. Reliability is part of the validity assessment.⁷-⁸,⁹ The validity testing of research instruments is intended to determine whether the study’s instruments can measure what should be measured.⁸,⁹,¹⁰,¹¹

The psychometric analysis is a method for testing construct validity, which is the extent to which an instrument can assess the construct of the intended measurement.¹² Construct validity was fundamental¹³ and conducted to determine whether the internal structure of the modified dental treatment motivation scale (DTMS) has been empirically proven. Reliability can be defined as the extent to which the measurement of a phenomenon provides stable and consistent results.⁸,⁹ Reliability also relates to repeatability.¹ Reliability testing is necessary because it refers to the consistency of integrated parts of the measuring instrument. For example, the scale is considered to have high internal consistency reliability if its items are “coherent” and measure the same construct.¹

The DTMS is an English language questionnaire that measures motivation to seek treatment based on the extrinsic and intrinsic motivation theory developed in India.¹⁴ This instrument will be used in Indonesia for the pregnant women’s population. Cross-cultural adaptation, validity, and reliability of the Indonesian version of the DTMS¹⁴ were conducted with pregnant women to determine whether this questionnaire can assess the motivation of pregnant women to seek dental health treatment, using validity, reliability, and adaptation for the Indonesian culture.

Pregnant women go through many physiological changes. These changes can be systemic and local, such as in the oral cavity. Oral health is an integral part of general health. Thus, oral problems faced by pregnant women must be addressed immediately.¹⁵ Indonesian pregnant women were chosen as respondents because 62.70% had never visited a dentist. Pregnant women receive oral health care to address their oral health complaints,¹⁶ but most of them, had never visited a dentist because they had not presented any oral health complaints.¹⁶ Oral manifestations have been more common among pregnant women,¹⁷ and very few know about these pregnancy-related changes and about the need to maintain good oral hygiene and to have regular dental examinations.¹⁶

Indonesian pregnant women were chosen as respondents because 62.70% had never visited a dentist. Pregnant women receive oral health care to address their oral health complaints,¹⁶ but most of them had never visited a dentist because they had not presented any oral health complaints.¹⁶ Oral manifestations have been more common among pregnant women¹⁷ and very few know about these pregnancy-related changes and about the need to maintain good oral hygiene and to have regular dental examinations¹⁶

Oral health treatment instruments for pregnant women can be considered the best scientific approach to gathering individual opinions about their experiences.¹⁸ Therefore, the motivation to seek treatment should be measured to allow tackling the problem in early pregnancy. Motivation has been shown to have physiological, behavioural, cognitive, and social foundations and it has played a fundamental role in optimising well-being, minimising physical pain, and maximising satisfaction. Motivation also plays an essential role in perceiving dental competence and treatment-seeking behaviour among dental patients.¹⁴

This introduction section refers to the importance of providing a valid and reliable measuring tool adapted for the Indonesian culture for the collection of information about the motivation of pregnant women to seek oral health treatment. By implementing such adapted measuring tool, knowledge of dental professionals, other health professionals, and community service providers will be improved; pregnant women’s oral health needs will be known and hopefully met; and optimal oral health care, education, support, and supervision will be provided. Therefore, this study aimed to obtain the results of psychometric analysis and reliability of the DTMS,¹⁴ which was adapted cross-culturally for pregnant women in Indonesia.

Methodology

The DTMS questionnaire was used in the present research, consisting of 15 items: seven questions (1, 2, 5, 7, 10, 13, and 15) about intrinsic motivation and eight questions (3, 4, 6, 8, 9, 11, 12, and 14) about extrinsic motivation.¹⁴ This scale measured the autonomous and controlled motivation to adopt a healthy attitude towards oral health care. The answers to the questions were classified according to a 5-point Likert scale, where 1 (minimum score) corresponded to “strongly disagree” and 5 (maximum score) meant “strongly agree.” The score for each dimension was obtained by adding all the answers to the items in each dimension. The mean score was obtained by dividing the total number of dimension scores by the maximum total dimension scores.¹⁴ Query items are shown in Table 1.

Table 1
DTMS treatment-seeking questionnaire

DTMS Treatment-seeking questionnaire
1. I feel that I want to take responsibility for my oral health
2. Others would be furious if I did not take the responsibility
3. I have carefully thought about the responsibility and believe that it is vital for many aspects of my life
4. My dentist asked me to take the responsibility
5. I believe that taking responsibility for my oral health is the best thing for me
6. I feel pressure from others to take the responsibility
7. I would feel guilty if I did not take the responsibility
8. I want others to approve of my decision
9. I want the dentist to think I am a good patient
10. It is easier to take the responsibility rather than to only think about it
11. I do not want others to be disappointed in me
12. Good oral health improves my social acceptability
13. I would feel bad about myself if I did not take responsibility for my oral health
14. I want others to see I can take the responsibility
15. It feels good to keep my oral cavity as clean as possible
Intrinsic motivation
Extrinsic motivation
Total motivation

In addition, a descriptive survey was conducted towards the first and second trimester of pregnancy at a maternal clinic, Banjaran, Bandung Regency, Indonesia, during the first and second weeks of December 2020. A total of 149 pregnant women were given information about the study and signed a consent form for their participation. All research procedures were performed in compliance with the Research Ethics Committee guidelines on studies involving human subjects, at the local, national, and regional levels, and also in conformity with the Declaration of Helsinki.¹⁹ The present research study was approved by the Health Research Ethics Committee of the Faculty of Medicine Universitas Padjadjaran, Indonesia, process no. 1108/UN6.KEP/EC /2020.

Population

Pregnant women who fulfilled the inclusion criteria and who were willing to participate in the study, and did not experience any pain were included. Pregnant women who were unable to communicate, read, and write well were excluded.

Sample size calculation

Sample size was estimated using a different scientific method based on either the ratio scale or statistical procedure that is specifically different from that of other study designs. Regarding exploratory factor analysis (EFA), it is recommended that the sample size should range from 100 to 250. The ratio scale and sample size estimation lacked any defin­itive ratio. Researchers use a minimum of two and a maximum of 20 people per item to estimate the sample size assumed arbitrarily.²⁰ The questionnaire consisted of 15 items to be answered by 300 individuals (maximum number of participants), and convenience sampling was used for sample size calculation. However, only 149 participants completed the questionnaire and the inclusion criteria were then met.

Cross-cultural adaptation

The questionnaire was adapted from English into Indonesian and followed the adaptation process recommended by WHO.⁶ The translation included the following steps: forward translation, expert panel back-translation, pre-testing, and cognitive interviews to obtain the final/modified version of the questionnaire.⁶

The DTMS questionnaire¹⁴ had content validity assessment initially through questionnaire translation by two linguists. They were public and private linguists²¹ – a Doctor of Arts in English Literature from Universitas Padjadjaran Centre of Language and a Bachelor of Arts in English Literature from a private English language school (Rainbow English Solution).

Four experts then analysed the translation results for face validity measurement: a Master of Public Health in Health Behaviour Studies, a Doctor of Public Health in Health Prevention Management Studies, a Doctor of Public Health in Health Behaviour Studies, and a Doctor of Psychology. The DTMS content validity was established by question correction. This improvement was made to make the questions clearer and to draw the final version of the modified DTMS questionnaire. The translation was re-checked by the four experts three days afterwards, and all of them approved of the translated version. The modified DTMS questionnaire was then back-translated into English.

The first pre-testing phase was conducted with 10 respondents by testing the Indonesian modified final version of the DTMS with the interview method. After filling in the questionnaire, the participants were interviewed about whether they had any difficulties filling in the questionnaire and whether they found any complicated and unclear questions. After being tested, the instrument was revised and referred to as the final/modified version of the questionnaire.

Validity and reliability

The psychometric analysis of the final/modified version of the questionnaire was performed through four validity test stages. The first stage consisted of the initial eigenvalue and cumulative percentage value assessments; the second stage included the EFA to determine the loading factor; the third stage consisted of the confirmatory factor analysis (CFA) to evaluate the questionnaire factor structure components; and the fourth stage included the goodness of fit index (GFI) assessment. The internal reliability analysis used Cronbach’s alpha and the test-retest was conducted using Pearson’s correlation coefficient.

Statistical analysis

Measurement was performed based on the COSMIN taxonomy (COnsensus-based Standards for the selection of health Measurement INstruments; HR-PRO and on the health-related patient-reported outcome of relationships of measurement properties).²² First, the construct validity test was carried out using the CFA test¹ by analysing the loading factor of the statement items taken from the Indonesian version of the modified DTMS. Furthermore, the significance and model fit tests were applied for advanced construct validity measurements. As a result, the statement item loading factor was categorised as valid if greater than 0.3 and significant if less than 0.05.

The construct validity analysis used the EFA, and the statement item distribution was tested using the CFA.¹²,²³,²⁴,²⁵ If it was greater than 1.96 at a 5% significance level, it would be considered valid.²⁴ The CFA expresses the degree of difference between the predicted and empirical factor structure of a variable and GFI.²⁶ Goodness of fit was measured by the chi-square test, CMN/DF, standardised RMR, root mean square error of approximation (RMSEA), comparative fit index (CFI), and GFI. The chi-square test of the goodness of fit was expected to be small. The CMN/DF ≤ 3.00, standardised RMR ≤ 0.08, RMSEA ≤ 0.05, CFI ≥ 0.96, and GFI ≥ 0.90.¹¹,²⁷ The validity test analysis was conducted using Lisrel 10 (Student Edition) and the SPSS version 25.²⁷

The reliability test was conducted using Cronbach’s alpha coefficient.¹ Reliability with minimally acceptable criteria greater than 0.5 was applied to measure the consistency of the items. Internal consistency was evaluated using Cronbach’s alpha and Pearson’s correlation coefficient. Cronbach’s alpha >0.7 was considered acceptable; >0.8 was considered good; and >0.9 was very good.¹¹ Pearson’s correlation coefficient is used to evaluate the test-retest reliability.⁵ This research hypothesised that the instrument would have good validity and reliability.

Results

This research was conducted with 149 pregnant women aged 18 to 44 years, with various educational levels (Table 2).

Table 2
Respondents’ Characteristics

Respondents’ Characteristics		N	%
1. Education level
	Primary (Elementary School – Junior High School)	54	29.53
	Secondary (Senior High School)	60	40.27
	Tertiary (Diploma Degree, Bachelor’s Degree, Master’s Degree, Doctorate Degree)	35	30.20
2. Age (in years)
	<20 years and ≥35 years	74	49.66
	20–34 years	75	50.33

Table 3
The Indonesian and English version of the DTMS Modification as Cross-Cultural Adaptation Result

No	Indonesian Statements	English Statements
C1	Saya merasa bahwa saya harus bertanggung jawab terhadap kesehatan gigi dan mulut saya setelah mendengar hasil dari wawancara dan pemeriksaan dokter gigi tadi	I feel that I have to be responsible for my dental and oral health after listening to the dentist’s explanation about my interview and examination result.
C2	Saya merasa ada orang lain yang akan memarahi saya jika saya tidak menjalani perawatan kesehatan gigi dan mulut berdasarkan hasil wawancara dan pemeriksaan dokter gigi tadi	I feel that others will be angry with me if I do not follow the treatment according to the results of my interview and examination.
C3	Saya sudah memikirkannya dengan sungguh-sungguh dan saya meyakini bahwa hasil perawatan kesehatan gigi dan mulut yang disampaikan dari hasil pemeriksaan dokter gigi tadi sangatlah penting bagi aspek-aspek kehidupan saya	I have considered and believed that my dental and oral examination results are necessary for aspects of my life.
C4	Dokter gigi saya menyuruh saya melakukan perawatan gigi dan mulut sesuai hasil wawancara dan pemeriksaan yang dilakukan dokter gigi tadi	My dentist tells me that I have to follow oral health treatment according to my interview and examination results.
C5	Saya meyakini bahwa setelah mendengar hasil wawancara dan pemeriksaan dokter gigi tadi, perawatan gigi dan mulut adalah yang terbaik bagi kesehatan gigi dan mulut saya	After receiving my interview and examination results, I believe that oral health treatment is best for my oral health status.
C6	Saya mendapat tekanan dari dokter gigi untuk menjalani perawatan gigi dan mulut dokter gigi tadi	The dentist forced me to follow the treatment.
C7	Saya akan merasa bersalah jika saya tidak menjalani perawatan gigi dan mulut yang telah diberitahukan hasil wawancara dan pemeriksaan dokter gigi tadi	I will feel guilty if I do not follow the treatment.
C8	Saya ingin agar orang lain setuju dengan perawatan gigi dan mulut yang saya lakukan	I want others to agree with my dental and oral treatment.
C9	Saya ingin agar dokter gigi menganggap saya sebagai pasien yang baik dengan mengikuti hasil pemeriksaan gigi dan mulut dokter gigi tadi	I want my dentist to believe that I am a good and compliant patient.
C10	Menurut saya lebih mudah menjalani daripada memikirkan perawatan gigi dan mulut berdasarkan hasil wawancara dan pemeriksaan dokter gigi tadi	I believe that following the treatment is more manageable than considering the dental and oral treatment, according to the results of my examination.
C11	Saya tidak ingin dokter gigi tadi merasa kecewa terhadap saya	I do not want the dentist to feel upset.
C12	Menurut saya tindakan pencarian pengobatan berdasarkan hasil pemeriksaan dokter gigi tadi akan membantu mempermudah penerimaan saya di masyarakat	I believe that treatment-seeking can help me with social acceptance.
C13	Saya akan merasa tidak enak terhadap diri saya sendiri jika saya tidak menjalani saran perawatan gigi dan mulut yang dokter gigi tadi	I will feel as if I am doing something wrong if I do not follow the treatment.
C14	Saya ingin orang lain melihat bahwa saya bisa menjalani perawatan gigi dan mulut yang disarankan oleh dokter gigi tadi	I want others to see that I can follow this dental and oral treatment as suggested by the dentist.
C15	Saya merasa nyaman jika saya bisa menjaga rongga mulut saya sebersih mungkin sesuai yang disampaikan oleh dokter gigi tadi	I feel comfortable if I can keep my oral cavity as clean as possible.

Respondents’ characteristics

The respondents’ characteristics are presented in Table 2. Research respondents were pregnant women who had completed secondary education, which reflects the schooling of Indonesian women as a whole, followed by primary education and tertiary education. The respondent’s ages ranged from <20 years and >35 years.

The construct validity analysis showed that the cumulative percentage was greater than 60% and had two components in its factor structure: intrinsic and extrinsic motivation (Table 4).

Table 4
Total variance explained for two DTMS components

Component	Initial Eigenvalue			Extraction sums of squared loading			Rotation sums of squared loading
	Total	% variance	% cumulative	Total	% variance	% cumulative	Total	% variance	% cumulative
1	11428	62.001	62.011	11428	62.011	62.011	11.394	61.827	61.827
2	1.767	9.590	71.601	1.767	9.590	71.601	2.040	13.600	54.483

Table 5 shows that the modification of DTMS was valid for measuring the dimensions of extrinsic and intrinsic motivation with loading factor value acquisition greater than 0.3. Statement item 6, however, was considered to have a negative relationship because the loading factor value was negative (-1.761 and 0.163).

Table 5
Results of the loading factor validity test based on the exploratory factor analysis (EFA) of the cross-cultural adaptation of the modified DTMS

No	Statements	Factor		Cutoff value	Notes
		1	2
C1	I feel that I have to be responsible for my dental and oral health after listening to the dentist’s explanation about my interview and examination result.		0.306	>0.3	Valid
C2	I feel that others will be angry with me if I do not follow the treatment according to the results of my interview and examination.		0.642	>0.3	Valid
C3	I have considered and believed that my dental and oral examination results are necessary for aspects of my life.	0.962		>0.3	Valid
C4	My dentist tells me that I have to follow dental and oral treatment according to my interview and examination results.	0.948		>0.3	Valid
C5	After receiving my interview and examination results, I believe that oral health treatment is best for my oral health status.		0.333	>0.3	Valid
C6	The dentist forced me to follow the treatment.		-0.613	<0.3	Not Valid
C7	I will feel guilty if I do not follow the treatment.		0.494	>0.3	Valid
C8	I want others to agree to my dental and oral treatment.	0.892		>0.3	Valid
C9	I want my dentist to believe that I am a good and compliant patient.	0.948		>0.3	Valid
C10	I believe that following the treatment is more manageable than considering the dental and oral treatment, according to the results of my examination.	0.461		>0.3	Valid
C11	I do not want the dentist to feel upset.	0.898		>0.3	Valid
C12	I believe that treatment-seeking can help me with social acceptance.	0.939		>0.3	Valid
C13	I will feel as if I am doing something wrong if I do not follow the treatment.	0.613		>0.3	Valid
C14	I want others to see that I can follow this dental and oral treatment as suggested by the dentist.	0.886		>0.3	Valid
C15	I feel comfortable if I can keep my oral cavity as clean as possible.	0.330		>0.3	Valid

The results showed that the statement items were grouped into two dimensions, as shown in the loading factor distribution (Figure 1). Figure 1 shows the median value as the statistical value of the loading factor test per question item, and the value in the right box is the significance value of the intrinsic and extrinsic motivation dimensions, which shows a significance value of p=0.00. Therefore, the correlation value between intrinsic and extrinsic motivation was -2.07. This result showed a significant correlation because the value was greater than 1.96 with =5%; the sign (–) indicates a negative correlation, which means that the higher the intrinsic motivation, the lower the extrinsic motivation.

Figure 1
Confirmatory factor analysis results of DTMS cross-cultural adaptation.

Figure 2 describes the mean value of the statistical significance value per statement item, and the value in the right box is the significance value of the intrinsic and extrinsic motivation dimensions; the margin of error was set by a p-value = 1.00. The correlation value between intrinsic and extrinsic motivation was -0.19, which was significant for the data provided by the 149 respondents. This result showed a significant correlation; a (-) sign indicated a negative correlation, which means that the higher the intrinsic motivation, the lower the extrinsic motivation.

Figure 2
Significance value of DTMS cross-cultural adaptation

The model testing results presented in Table 6 show that the test values for the six modelling items are in line with the expected test values. The goodness of fit value was greater than or equal to 0.90 (0.903).

Table 6
Testing results for the DTMS modification model

Goodness of fit criteria	Test value	Expected value	Notes
Chi-square	125.804 (p=0.0021)	Expected to be low	Compliant
CMN/DF	1.498	≤3.00	Compliant
Standardised RMR	0.0702	≤0.08	Compliant
Root mean square error of approximation (RMSEA)	0.0578	≤0.05	Compliant
Comparative fit index (CFI)	0.976	≥0.96	Compliant
Goodness of fit index (GFI)	0.903	≥0.90	Compliant

The internal consistency reliability measurement results are listed in Table 7. Cronbach’s alpha intrinsic motivation value standard was set at 0.737, while the extrinsic motivation was set at 0.933. Therefore, the internal consistency reliability in Table 7 suggests that Cronbach’s alpha value of 14 statements showed reliable results. Conversely, statement C6 showed an unreliable Cronbach’s alpha value of 0.972, which should have been lower than 0.933.

Table 7
Internal consistency reliability of the modified DTMS (n=34)

No	Statements	Cronbach’s alpha	Notes
Intrinsic motivations
C1	I feel that I have to be responsible for my dental and oral health after listening to the dentist’s explanation about my interview and examination result.	0.715	Reliable
C2	I feel that others will be angry with me if I do not follow the treatment according to the results of my interview and examination.	0.755	Reliable
C5	After receiving my interview and examination results, I believe that oral health treatment is best for my oral health status.	0.709	Reliable
C7	I will feel guilty if I do not follow the treatment.	0.684	Reliable
C10	I believe that following the treatment is more manageable than considering the dental and oral treatment, according to the results of my examination	0.695	Reliable
C13	I will feel as if I am doing something wrong if I do not follow the treatment.	0.678	Reliable
C15	I feel comfortable if I can keep my oral cavity as clean as possible	0.711	Reliable
Extrinsic motivations
C3	I have considered and believed that my dental and oral examination results are necessary for aspects of my life.	0.915	Reliable
C4	My dentist tells me that I have to follow dental and oral treatment according to my interview and examination results.	0.915	Reliable
C6	The dentist forced me to follow the treatment.	0.972	Unreliable
C8	I want others to agree to my dental and oral treatment.	0.917	Reliable
C9	I want my dentist to believe that I am a good and compliant patient.	0.914	Reliable
C11	I do not want the dentist to feel upset.	0.917	Reliable
C12	I believe that treatment-seeking can help me with social acceptance.	0.912	Reliable
C14	I want others to see that I can follow this dental and oral treatment as suggested by the dentist.	0.918	Reliable

The test-retest results showed that the respective correlation values were within the range of 0.985-0.990, which is considered reliable (Table 8). The correlation value showed a p-value <0.05 in the first and second tests of the intrinsic and extrinsic motivation in the questionnaire.

Table 8
Test-retest results for the modified DTMS (n=68)

Motivations	Test	Test results		Notes
		Intrinsic_1	Intrinsic_2
Intrinsic_1	Pearson’s correlation		0.985	Reliable
	P-value		0.000
	N	68	68
Intrinsic_2	Pearson’s correlation	0.990		Reliable
	P-value	0.000
	N	68	68
		Extrinsic_1	Extrinsic_2
Extrinsic_1	Pearson’s correlation	1	0.990	Reliable
	P-value		0.000
	N	68	68
Extrinsic_2	Pearson’s correlation	0.990		Reliable
	P-value	0.000
	N	68	68

Discussion

Most respondents had secondary education (Table 2). This is similar to the study by Sari et al.,²⁸ in which Malaysian pregnant women had secondary education. This sociodemographic aspect can influence research results. Several measures of socioeconomic background are used in epidemiological studies. Maternal education has one of the most widespread impacts on motivation; maternal behavioural factors tend to be more closely related to behavioural health education attainment, which is affected evenly in high-income countries with high levels of human development and gender equality.⁸

The ages of most respondents were not <20 years and ≥35 years in the current research, but 20-35 years (Table 2). The age group classification was based on groups of pregnant women who were at higher and lower risk. As suggested by the findings of Azofeifa et al.,²⁹ who showed most pregnant women in their study were aged 15-24 years and had less than secondary education, and those from racial minority groups reported lower rates for dental hygiene visits, as compared to non-pregnant women. Pregnant women aged >35 years are rarely found because of the many risks associated with pregnancy at that age.³⁰ Also, pregnant women aged >35 years, categorised as advanced maternal age, often feel that they already have too much information about pregnancy risks associated with their age and foetal disorders, thus making them anxious and making it difficult for them to focus on positive outcomes. Nonetheless, these women still want to get as much information as possible.³⁰

The assessment of the construct validity initiated with a translation from English into Indonesian led to the adaptation of the DTMS questionnaire (Table 1). Two linguists from public and private institutions translated and adapted the questionnaire. This process was in line with WHO’s guidelines and with the research conducted by Beyera et al.,²¹ in which the questionnaire was translated by two independent translators. The purpose of translation by two translators is to provide reliable equivalence.⁶,²¹

The translation results were then reviewed through a focus group discussion, consisting of a Master of Public Health in Health Behaviour Studies, a Doctor of Public Health in Health Prevention Management Studies, a Doctor of Public Health in Health Behaviour Studies, and a Doctor of Psychology with expertise in intrinsic and extrinsic motivations. This process was consistent with WHO’s adaptation guidelines and with the research conducted by Beyera et al.⁶,²¹ This whole process was aimed to assess the appropriateness of content validity.¹¹,²¹

Adaptation and psychometric analysis of the DTMS was the first to be carried out in Indonesia. One hundred forty-nine research respondents can be considered sufficient because, as stated by Kyriazos³¹, a sample size of 100 or less is certainly too small, and a sample size of 100–200 may be sufficient for the EFA.³² Furthermore, pregnant women were chosen as respondents of the current research regarding the cross-cultural adaptation, psychometric analysis, validity, and reliability of the DTMS questionnaire because the examination results of the modified questionnaire were expected to support the questionnaire’s ability to correctly assess the motivation of pregnant women in seeking dental health treatment.

Figure 1 presents the mean value as the statistical test value for factor loading per statement item. The value in the right box is the significance value of the intrinsic and extrinsic motivation dimensions, which showed a significance value of p=0.00; the correlation value between intrinsic and extrinsic motivation was -2.07. These values showed a significant correlation (greater than 1.96, with α=5%). A (-) sign indicated a negative correlation, which means that the higher the intrinsic motivation dimensions, the lower the extrinsic dimensions. Figure 2 shows the correlation value between intrinsic and extrinsic motivation was -0.19 and considered significant for the data provided by 149 respondents. These statistical values showed significant correlation, with the (-) sign. This result corroborates the previous explanation for the findings. Both figures also indicated that the questionnaire of the current research is valid to assess intrinsic and extrinsic motivation.

Table 4 shows that the psychometric analysis of construct validity using the EFA resulted in cumulative percentage values greater than 60% and evaluated two components, namely intrinsic and extrinsic motivations. This result was consistent with the study conducted by Pinxten et al.,³² whose construct validity demonstrated cumulative percentage variance greater than 60% and evaluated two components. A cumulative percentage variance greater than 60% is quite good because the cutoff point is around 60% and assesses two factors,³³ which follows the intrinsic and extrinsic motivation theory, on which the DTMS is based.¹⁴ Construct validity is a theoretical validity that assesses the framework of measurement variables. An instrument is said to have construct validity if the items are organised into query items, which will measure every factor taken into account.¹⁰

The validity test results for the motivation of pregnant women to seek oral health treatment using the EFA are presented in Table 4, which shows that the instrument was proven to be valid in measuring the factor items categorized into two dimensions. This result was similar to that obtained by Nagarajan et al.,¹⁴ who suggested that the two dimensions in their findings were extrinsic and intrinsic motivation. used in the DTMS questionnaire.

The loading factor value obtained from the cross-cultural adaptation of the DTMS was greater than 0.3. A value greater than 0.3 is the reference value for determining the construct validity using the EFA.³³ A total of 14 statement items indicated a loading factor greater than 0.3, which was considered valid. The results above were obtained from the EFA, and the rotation sums of squared loadings was performed. These results were consistent with those of the psychometric research conducted by Pinxten et al.,³² who found a valid loading greater than 0.3. However, statement 6 was considered not valid because the loading factor value was -0.163 (less than 0.3). This result might have been caused by the stress the respondents went through. The statement was, “I was under pressure from the dentist to undergo dental treatment”.

The test value of six modelling items from the test model (Table 6) was 0.903. This value was consistent with the expected test value and was based on the accepted reference value for a model.²⁹,³⁰ These results indicated a good fit model, in which the theory and its implementation were measured synchronously and precisely. These results were also similar to those of the research conducted by Azofeifa et al.²⁹ and Goisis et al.,³⁰ who stated that six of the eight criteria for the goodness of fit were met. Thus, the statement items of the modified instrument in the present study can also be considered valid.²⁹,³⁰ Therefore, the fundamental theory was proven to be the same with the measurement of the two motivation dimensions.

The EFA was then followed by the CFA. The results for the CFA for the distribution of the question items showed that the statement items were grouped into two dimensions, as shown in the loading factor distribution description (Figure 1). The significance value of the statement items (0.00215) is listed in Figure 2. This result was quite similar to that of several studies, such as the research conducted by Pinxten et al.³² and Kyriazos³¹, which stated that the EFA and the CFA are factor analyses that serve as a diagnostic tool to evaluate whether the collected data are in line with the theoretically expected pattern or structure of the target construction.³¹-³²,³⁴

Table 8 shows the good correlation values of the modified DTMS in the first and the second tests. These results indicate that the respondents consistently answered the questionnaire; thus, this questionnaire was reliable for pregnant women in Indonesia. Fourteen statements had a Cronbach’s alpha greater than 0.737, except for one statement: statement C6, which had a Cronbach’s alpha greater than 0.933 (0.972). This value was almost similar to the reliability test results obtained by Pinxten et al. (32); in their study, Cronbach’s alpha was 0.986. This value indicated excellent reliability.²³,³³ The test-retest results suggested a value in the range of 0.985–0.990. This score also indicated good internal reliability and test-retest results due to the high internal consistency score (α=0.83 - 0.97) and the strong correlation between the overall score and the subscale score (r=0.78-0.99).³⁴ This value also indicated good reliability.³³ These results indicated that respondents answered the questionnaire consistently; thus, the questionnaire in the current research, adapted to be used with pregnant women, was considered reliable. Reliability refers to how the results obtained by specific measurements and procedures can be replicated. Reliability makes an essential contribution to questionnaire validity. Thus, the reliability test should be performed and generate good results.¹¹

The findings of the present study provided preliminary support for the reliability and validity of the cross-cultural adaptation and translation of the DTMS. The translation and validation processes estimated the cross-cultural equivalence of the instrument. The high Cronbach’s alpha value and the item-total correlation obtained provide evidence of the reliability of the cross-cultural adaptation and translation of the DTMS.

The Indonesian adapted version of the DTMS was proven valid and reliable in 14 out of 15 statements, but considered not valid in statement 6. The EFA and the CFA test values indicated a valid result in the measure construction theory of the Indonesian adapted version of the DTMS. The test model showed a CFA testing model with a good fit, which measured the application of the theory and its implementation synchronously and precisely. The internal reliability test and test-retest results also showed that the modified questionnaire was reliable in assessing intrinsic and extrinsic motivation and that it can therefore be used in Indonesia.

The Indonesian adapted version of the DTMS is expected to measure the motivation of pregnant women to seek oral health treatment before lesions occur in hard or soft tissues in the oral cavity as a result of hormonal changes. Furthermore, it is hoped that this instrument can be used in conjunction with a health measurement instrument that evaluates the effects of disease and treatment from time to time;³⁵ thus, the motivation and evaluation results obtained for health services can be clearly described.

Study limitations

This study aimed to validate the instrument for use in the Indonesian population; therefore, the findings should not be extrapolated to other countries.

Conclusion

The Indonesian version of the DTMS proved to be a reliable and valid instrument to measure the motivation of Indonesian pregnant women to seek oral health treatment. Cross-cultural adaptation and psychometric analysis of the Indonesian version of the DTMS proved to be successful and conceptually suitable for the Indonesian population. The EFA and the CFA were valid for the measure theoretical construction. In addition, internal reliability and test-retest values show the reliability of the questionnaire, with high internal consistency in assessing intrinsic and extrinsic motivation, allowing it to be used for the Indonesian culture and population.

Acknowledgements

We would like to thank Universitas Padjadjaran for partly funding this current research through the Academic Leadership Grant (ALG) for the research program of Universitas Padjadjaran (grant no. 1427/UN6.3.1/LT/2020).

References

• ¹
  Taherdoost H. Validity and Reliability of the Research Instrument; How to Test the Validation of a Questionnaire/Survey in a Research. SSRN Electron J. 2018;(January 2016).
• ²
  Suzy A, Amriwijaya J, Fitriana E. Trans-adapted, reliability, and validity of children fear survey schedule-dental subscale in Bahasa Indonesia. Dent J (Majalah Kedokt Gigi). 2015;48(1):1.
• ³
  Gjersing L, Rm Caplehorn J, Clausen T. Cross-cultural adaptation of research instruments: language, setting, time and statistical considerations [Internet]. 2010 [cited 2021 Jun 25]. Available from: http://www.biomedcentral.com/1471-2288/10/13
  » http://www.biomedcentral.com/1471-2288/10/13
• ⁴
  Epstein J, Santo RM, Guillemin F. A review of guidelines for cross-cultural adaptation of questionnaires could not bring out a consensus. [cited 2021 Jun 25]; Available from: http://dx.doi.org/10.1016/j.jclinepi.2014.11.021
  » http://dx.doi.org/10.1016/j.jclinepi.2014.11.021
• ⁵
  Abdulameer SA, Sahib MN. The Open Rheumatology Journal Cross-Cultural Adaptation and Psychometric Properties of Osteoporosis Knowledge Tool-Arabic Version Among Iraqi Population. [cited 2021 Jun 25]; Available from: https://openrheumatologyjournal.com
  » https://openrheumatologyjournal.com
• ⁶
  Sousa VD, Rojjanasrirat W. Translation, adaptation and validation of instruments or scales for use in cross-cultural health care research: A clear and user-friendly guideline. J Eval Clin Pract. 2011;17(2):268–74.
• ⁷
  Sullivan GM. A Primer on the Validity of Assessment Instruments. Available from: http://meridian.allenpress.com/doi/pdf/10.4300/JGME-D-11-00075.1
  » http://meridian.allenpress.com/doi/pdf/10.4300/JGME-D-11-00075.1
• ⁸
  Suwargiani AA, Mustika I, Pribadi S, Hidayat W, Jasrin A. Validation and reliability of Oral Health Impact Profile-14 questionnaire among pregnant woman (Anne S, et al) Validation and realibility of Oral Health Impact Profile-14 questionnaire among pregnant woman [Internet]. Vol. 28, Padjadjaran Journal of Dentistry. 2016 Jul [cited 2021 Jun 25]. Available from: https://jurnal.unpad.ac.id/pjd/article/view/13725
  » https://jurnal.unpad.ac.id/pjd/article/view/13725
• ⁹
  Maulina T, Nadiyah Ridho S, Asnely Putri F. Validation of Modified Dental Anxiety Scale for Dental Extraction Procedure (MDAS-DEP). Open Dent J. 2019 Nov 16;13(1):358–63.
• ¹⁰
  Said H, Bakri Badru B. Confirmatory Factor Analysis (Cfa) for Testing Validity And Reliability Instrument in the Study of Education. Aust J Basic Appl Sci. 2011;5(12):1098–103.
• ¹¹
  Akeem Bolarinwa O. Principles and Methods of Validity and Reliability Testing of Questionnaires Used in Social and Health Science Researches. 2015 [cited 2021 Jun 25]; Available from: www.npmj.org
  » www.npmj.org
• ¹²
  Richardson J, Iezzi A, Khan MA, Chen G, Maxwell A. Measuring the Sensitivity and Construct Validity of 6 Utility Instruments in 7 Disease Areas. Available from: http://www.sagepub.com/journalsPermissions.nav
  » http://www.sagepub.com/journalsPermissions.nav
• ¹³
  Prinsen CAC, Mokkink LB, Bouter LM, Alonso J, Patrick DL, de Vet HCW, et al. COSMIN guideline for systematic reviews of patient-reported outcome measures. Qual Life Res. 2018;27(5):1147–57.
• ¹⁴
  Nagarajan S, Reddy C, Chandra RV. Motivation in periodontal therapy: Assessment using novel Dental Treatment Motivation Scale (DTMS). Dentistry. 2014;4(10):1–4.
• ¹⁵
  Favero V, Bacci C, Volpato A, Bandiera M, Favero L, Zanette G. Pregnancy and dentistry: A literature review on risk management during dental surgical procedures. Dent J [Internet]. 2021 [cited 2021 Dec 11];9(46):1–16. Available from: https://doi.org/10.3390/dj9040046
  » https://doi.org/10.3390/dj9040046
• ¹⁶
  Suwargiani AA, Arief EM, Aripin D, Widyaputra S, Susilawati S. Oral health care practice of women with pregnancy experience. Padjadjaran J Dent J Dent [Internet]. 2020 Nov 30 [cited 2021 Jun 15];32(3):197–206. Available from: http://jurnal.unpad.ac.id/pjd/article/view/30312
  » http://jurnal.unpad.ac.id/pjd/article/view/30312
• ¹⁷
  Hidayat W, Mustika I, Pribadi S, Zakiawati D, Suwargiani AA. Profile of oral manifestations, oral hygiene, and nutritional status in pregnant women. Padjadjaran J Dent [Internet]. 2019 Nov 30 [cited 2021 Jun 15];31(3):215–9. Available from: http://jurnal.unpad.ac.id/pjd/article/view/23792
  » http://jurnal.unpad.ac.id/pjd/article/view/23792
• ¹⁸
  Fakheran O, Keyvanara M, Saied-Moallemi Z, Khademi A. The impact of pregnancy on women’s oral health-related quality of life: A qualitative investigation. BMC Oral Health [Internet]. 2020 Dec 1 [cited 2021 Dec 11];20(1):1–11. Available from: https://doi.org/10.1186/s12903-020-01290-5
  » https://doi.org/10.1186/s12903-020-01290-5
• ¹⁹
  World Medical Association Declaration of Helsinki: Ethical principles for medical research involving human subjects. JAMA. 2013 Nov 27;310(20):2191–4.
• ²⁰
  Anthoine E, Moret L, Regnault A, Sbille V, Hardouin JB. Sample size used to validate a scale: A review of publications on newly-developed patient reported outcomes measures. Health Qual Life Outcomes. 2014;12(1):1–10.
• ²¹
  Kebede G, Id B, O’Brien J, Campbell S. The development and validation of a measurement instrument to investigate determinants of health care utilisation for low back pain in Ethiopia. 2020; Available from: https://doi.org/10.1371/journal.pone.0227801
  » https://doi.org/10.1371/journal.pone.0227801
• ²²
  Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL, et al. The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes. J Clin Epidemiol. 2010;63(7):737–45.
• ²³
  Alumran A, Hou XY, Hurst C. Validity and reliability of instruments designed to measure factors influencing the overuse of antibiotics. J Infect Public Health [Internet]. 2012;5(3):221–32. Available from: http://dx.doi.org/10.1016/j.jiph.2012.03.003
  » http://dx.doi.org/10.1016/j.jiph.2012.03.003
• ²⁴
  Fitriana E. Confirmatory Factor Analysis of the Bandung Family Relation Test: A Simulation Study Comparing ML, DWLS and WLS Estimation. Nijmegen; 2015.
• ²⁵
  Brown TA. Confirmatory Factor Analysis for Applied Research [Internet]. 2nd ed. New York: Guilford Publication; 2015 [cited 2021 Jun 25]. 88 p. Available from: https://www.routledge.com/Confirmatory-Factor-Analysis-for-Applied-Research/Brown/p/book/9781462515363#
  » https://www.routledge.com/Confirmatory-Factor-Analysis-for-Applied-Research/Brown/p/book/9781462515363#
• ²⁶
  Prudon P. Confirmatory factor analysis as a tool in research using questionnaires: a critique. 2015; Available from: www.AmmonsScientific.com
  » www.AmmonsScientific.com
• ²⁷
  Kusbiantari D, Fitriana E, Hinduan ZR, Srisayekti W. The Open Psychology Journal Psychometric Properties of Binge Eating Scale Indonesian Version. [cited 2021 Jun 25]; Available from: https://openpsychologyjournal.com
  » https://openpsychologyjournal.com
• ²⁸
  Sari EY, Saddki N, Yusoff A. Association between perceived oral symptoms and presence of clinically diagnosed oral diseases in a sample of pregnant women in Malaysia. Int J Env Res Public Heal [Internet]. 2020;17(19):7337. Available from: www.mdpi.com/journal/ijerph
  » www.mdpi.com/journal/ijerph
• ²⁹
  Azofeifa A, Yeung LF, Alverson CJ, Beltrán-Aguilar E. Dental caries and periodontal disease among U.S. pregnant women and nonpregnant women of reproductive age, National Health and Nutrition Examination Survey, 1999–2004. J Public Health Dent [Internet]. 2016 Sep 1 [cited 2021 Jun 25];76(4):320–9. Available from: /pmc/articles/PMC5097890/
• ³⁰
  Goisis A, Remes H, Barclay K, Martikainen P, Myrskylä M. Original Contribution Advanced Maternal Age and the Risk of Low Birth Weight and Preterm Delivery: a Within-Family Analysis Using Finnish Population Registers. 2017;186(11). Available from: https://academic.oup.com/aje/
  » https://academic.oup.com/aje/
• ³¹
  Kyriazos TA. Applied Psychometrics: Sample Size and Sample Power Considerations in Factor Analysis (EFA, CFA) and SEM in General. Psychology. 2018;09(08):2207–30.
• ³²
  Pinxten WJL, Fitriana E, De Jong C, Klimas J, Tobin H, Barry T, et al. Excellent reliability and validity of the Addiction Medicine Training Need Assessment Scale across four countries. J Subst Abuse Treat. 2019;99(October 2018):61–6.
• ³³
  Schmits E, Heeren A, Quertemont E. The self-report version of the LSAS-CA: Psychometric properties of the French version in a non-clinical adolescent sample. Psychol Belg [Internet]. 2014 Feb 25 [cited 2021 Jun 25];54(2):181–98. Available from: http://dx.doi
  » http://dx.doi
• ³⁴
  Maydeu-Olivares A. Goodness-of-Fit Assessment of Item Response Theory Models. Measurement. 2013;11(3):71–101.
• ³⁵
  Terwee CB, Prinsen CAC, Ricci Garotti MG, Suman A, de Vet HCW, Mokkink LB. The quality of systematic reviews of health-related outcome measurement instruments. Qual Life Res. 2016;25(4):767–79.

Publication Dates

• Publication in this collection
  08 July 2022
• Date of issue
  2022

History

• Received
  13 July 2021
• Reviewed
  14 Apr 2022
• Accepted
  08 Mar 2022