Cross-cultural psychometric assessment of an appetite questionnaire for patients with cancer

Avaliação psicométrica transcultural de um questionário de apetite para pacientes com câncer

Abstract

Objective

To evaluate the psychometric properties, along with cross-cultural invariance analysis, of the Cancer Appetite and Symptom Questionnaire (CASQ).

Method

Data from 555 United Kingdom (UK) cancer patients were used to evaluate the psychometric properties of the CASQ. Construct validity was assessed through factorial and convergent validity. We conducted a confirmatory factor analysis using as indices the chi-square ratio by degrees of freedom (χ²/df), the comparative fit index (CFI), the goodness of fit index (GFI), and the root mean square error of approximation (RMSEA). Convergent validity was estimated by the items’ average variance extracted (AVE). Reliability was estimated by composite reliability and internal consistency. Factorial invariance analysis of the CASQ was evaluated by multigroup analysis (∆χ²) using the UK and Brazilian samples.

Results

All items showed adequate psychometric sensitivity in the UK sample. One item was removed and four correlations were included between errors with an appropriate fit of the model (χ²/df = 2.674, CFI = 0.966, GFI = 0.964, RMSEA = 0.055). The reliability of the CASQ was adequate and the convergent validity was low. The factorial structure of the CASQ differed across countries, and a lack of measurement invariance for the two countries was observed (λ: ∆χ² = 64.008, p < 0.001; i: ∆χ² = 3515.047, p < 0.001; Res: ∆χ² = 4452.504, p < 0.001).

Conclusion

The CASQ showed adequate psychometric properties in the UK sample. The ability to estimate loss of appetite and the presence of symptoms was different between UK and Brazilian patients.

Keywords:
Cross-cultural study; cancer; symptom; appetite

Resumo

Objetivos

Avaliar as propriedades psicométricas, juntamente com a análise de invariância transcultural, do Questionário de Apetite e Sintoma para Pacientes com Câncer (Cancer Appetite and Symptom Questionnaire, CASQ).

Métodos

Dados de 555 pacientes com câncer do Reino Unido foram utilizados para avaliar as propriedades psicométricas do CASQ. A validade de construto foi estimada por meio das validades fatorial e convergente. Realizou-se análise fatorial confirmatória utilizando como índices a razão de qui-quadrado pelos graus de liberdade (χ²/gl), o comparative fit index (CFI), o goodness of fit index (GFI) e o root mean square error of aproximation (RMSEA). A validade convergente foi estimada pela variância extraída média (VEM). A confiabilidade foi estimada pela confiabilidade composta e consistência interna. A análise de invariância fatorial do CASQ foi avaliada por análise multigrupos (∆χ²) usando as amostras do Reino Unido e do Brasil.

Resultados

Todos os itens apresentaram adequada sensibilidade psicométrica na amostra do Reino Unido. Um item foi removido e foram incluídas quatro correlações entre erros, o que resultou em ajustamento adequado do modelo à amostra (χ²/df = 2,674, CFI = 0,966, GFI = 0,964, RMSEA = 0,055). A confiabilidade do CASQ foi adequada e a validade convergente foi baixa. A estrutura fatorial do CASQ diferiu entre os países, e uma falta de invariância foi observada para os dois países (λ: ∆χ² = 64,008, p < 0,001; i: ∆χ² = 3515,047, p < 0,001; Res: ∆χ² = 4452,504, p < 0,001).

Conclusão

O CASQ apresentou adequadas propriedades psicométricas na amostra do Reino Unido. A capacidade de estimar a falta de apetite e a presença de sintomas foi diferente entre pacientes do Reino Unido e do Brasil.

Descritores:
Estudo transcultural; câncer; sintoma; apetite

Introduction

Addressing the symptoms caused by cancer and its treatment is an important priority in the clinical management of patients. ^1–3 In particular, decreased appetite has been linked to weight loss and the exacerbation of cachexia. ^4,5 Cachexia, in turn, has a significant impact on quality of life and patient survival. ⁶

The measurement of appetite is a challenge for clinicians, since there are many aspects involved that need to be considered at the time of evaluation. Wilson et al., ⁶ in this context, proposed the Council on Nutrition Appetite Questionnaire (CNAQ) which has since been developed into the Cancer Appetite and Symptom Questionnaire (CASQ). ⁷ The CASQ dimensions cover appetite and symptoms such as satiety, taste, eating pattern, mood, and disease-related symptoms such as nausea and pain. This instrument, which is predictive for weight loss, was proposed in the English language, and its reliability, content and predictive validity were evaluated and documented in a sample of 185 patients with cancer in the United Kingdom (UK). ⁷ Subsequently, the cross-cultural adaptation of CASQ into Portuguese was completed; its content and construct (factorial and convergent) validity and its reliability were assessed with a sample of 1,140 Brazilian patients with cancer. ⁸

Given that the validity of an instrument depends on the sample it is applied to and not on the instrument itself, ^9,10 evaluation of the psychometric properties of the CASQ, in terms of validity and reliability, is imperative to determine the quality of the information obtained and the instrument’s ability to capture the construct studied. Thus, the same instrument can capture the construct differently when applied to samples with different demographic, social, and/or cultural characteristics. For this reason, the psychometric properties of an instrument must be evaluated for each sample prior to comparing their results. In addition, when using the same instrument in different countries, there is a need to perform the steps of cross-cultural adaptation and stability evaluation for the instrument. For that purpose, transnational studies are required.

Thus, this study was performed to evaluate the psychometric properties of the CASQ for patients with cancer when applied to a sample of patients from the UK, along with invariance analysis when compared to a Brazilian sample.

Method

Study design and sample size

This was a cross-sectional study. The minimum sample size was established using the recommendations by Hair et al., ¹¹ where a minimum of five to ten subjects per model parameter (k) should be investigated to evaluate the psychometric quality of an instrument. Thus, using α = 5%, β = 20%, k = 10, and considering that the CASQ has 24 parameters to be estimated, we calculated that the sample size should have at least 240 subjects, which our dataset exceeded.

Participants

United Kingdom

Participants were patients with cancer attending Nottingham University Hospitals NHS Trust oncology or palliative care services. The majority of adult patients completed the CASQ as part of their routine clinical evaluation by a dedicated rehabilitation service (n = 370) ¹² ; the remainder (n = 185) were participants in the initial validity testing of the CASQ. ⁷

Brazil

Participants were adult patients (18 years and above) who were attending Hospital de Câncer de Barretos – Fundação Pio XII, in Barretos, state of São Paulo, with a diagnosis of cancer.

Exclusion criterion

Those undergoing major and intermediate complex surgical procedures, showing cognitive impairment, severe psychiatric disorders or edema were excluded. These exclusion criteria were used in both the UK and Brazil.

Procedures

Participants from the UK self-completed the CASQ based on how they had felt over the last day or two. In Brazil, the CASQ was completed through interview. Weight and height were assessed by the clinical team for the UK participants. These measurements were self-reported by the Brazilian participants.

Measurement instrument

The CASQ was originally proposed in the English language ⁷ with a one-factor model. The instrument consists of 12 items with Likert responses of five points and four items that were formulated in the opposite direction of the remaining eight. In this study, we used the original version of the instrument and the Portuguese version developed by our research group. ⁸

Psychometric properties

The psychometric properties of the CASQ were estimated for the UK sample by evaluating psychometric sensitivity and construct-related validity. These properties have been estimated previously for the Brazilian sample. ⁸

Data analysis: psychometric qualities

Sensitivity of items

The psychometric sensitivity of items was evaluated through measures of central tendency (mean, median, mode), variability (standard deviation), and shape distribution (skewness and kurtosis). The sensitivity was considered appropriate when absolute values of skewness and kurtosis were < 3 and <7, respectively, ¹³ thus indicating absence of severe deviation from normality.

Construct validity

Construct validity was assessed by factorial and convergent validity.

Factorial validity

We used confirmatory factor analysis (CFA) with maximum likelihood estimation. As indices for assessing the goodness of the model fit to the sample, we used the chi-square ratio by degrees of freedom (χ²/df), the comparative fit index (CFI), the goodness of fit index (GFI), and the root mean square error of approximation (RMSEA). ^11,13 The fit model was considered appropriate when χ²/df ≤ 2.0, CFI and GFI ≥ 0.9, and RMSEA ≤ 0.08. ^11,13

The items that showed factorial weights (λ) < 0.30 were removed. Modification indices, calculated by Lagrange multipliers (LM), were used to further improve the model fit. Correlations between errors of the items were entered when LM > 11. ¹³

Comparison between the original model and the model refined using the above stated criteria (λ and LM) was performed using information theory indices (Akanke information criterion [AIC], Bayesian information criterion [BIC], and Browne-Cudeck criterion [BCC]). The best model was the one that showed the lowest values on one or more of these indices. ¹³

Convergent validity

Convergent validity occurs when the items that are reflections of a factor are heavily saturated on this factor. It can be estimated by using the items’ average variance extracted (AVE) by their respective factors. ¹⁴ Convergent validity was considered adequate when AVE ≥ 0.50. ¹⁴

Reliability

The reliability of the CASQ was assessed by composite reliability (CR) and internal consistency. CR was estimated with the Fornell & Larcker ¹⁴ technique, and internal consistency by the standardized Cronbach’s alpha coefficient (α), which were considered adequate when CR and α ≥ 0.70.

Invariance UK vs. Brazil

The invariance of the CASQ model when applied to the UK and Brazilian samples was evaluated by multigroup analysis using the chi-square difference statistic (∆χ²). The following variables were assessed: i) factor weights (λ) (metric invariance/weak invariance); ii) factor weights (λ) and item intercepts (i) (scalar invariance/strong invariance); iii) factor weights (λ), item intercepts (i) and residual variances/covariances (residuals invariance/strict invariance) (Res). ¹⁵ The model was considered invariant when ∆χ² p > 0.05. The explained variance by the model, i.e., the proportion of item variances on mean items due to the latent factor (appetite and symptoms) in the different samples was also calculated.

Ethical considerations

Data collection in Brazil was approved by the human research ethics committee of Hospital de Câncer de Barretos (protocol 561/2011). The initial CASQ validation study was approved by the Derbyshire local research ethics committee (protocol 07/H0401/92). Only patients who agreed and signed the informed consent form participated in the study.

Results

Data from 1,140 patients in Brazil and 555 patients in the UK were included in the analysis. Table 1 shows the sample characteristics.

Table 1
Characteristics of Brazilian and UK participants

Characteristic		Brazil		United Kingdom
		n	%	n	%
Gender
	Male	441	38.7	334	60.2
	Female	699	61.3	221	39.8
Diagnosis
	Head and neck	79	6.9	-	-
	Upper digestive tract	108	9.5	-	-
	Lower digestive tract	222	19.5	-	-
	Gynecology	135	11.8	-	-
	Hematology	5	0.4	-	-
	Breast cancer	335	29.4	-	-
	Brain tumor	17	1.5	-	-
	Orthopedics	27	2.4	-	-
	Skin	46	4.0	-	-
	Thorax *	66	5.8	555	100.0
	Urology	100	8.8	-	-
Treatment
	In treatment	1,016	89.0	152	27.4
	Palliative care	120	11.0	403	72.6
Body mass index (kg/m2)
	< 18.5 (underweight)	78	6.9	49	8.8
	18.5├25.0(healthy weight)	468	41.6	270	48.6
	25.0├30.0 (overweight)	363	32.3	170	30.6
	≥ 30.0 (obese)	215	19.1	62	11.2

• ^*
  Lung, pleura and mediastinum.

The mean age of participants from the UK and Brazil at the time of completion of the CASQ was 70.00 (SD = 9.70) and 53.95 (SD = 13.25) years, respectively. Mean body mass index was 24.45 kg/m² (SD = 4.83) for UK patients and 25.80 kg/m² (SD = 5.54) for Brazilians patients. It can be observed that the samples from the two countries have different characteristics.

Table 2 presents the summary of measures obtained for the items in the original version of the CASQ for patients with cancer in the UK.

Table 2
Summary of measures of component items of the original version of the CASQ for patients with cancer in the United Kingdom

CASQ	Mean	Median	Mode	SD	Skewness	Kurtosis
Item 1	2.10	2	2	1.15	-0.04	-0.80
Item 2	3.06	3	4	1.05	-1.14	0.64
Item 3	1.71	2	2	1.15	-0.09	-0.96
Item 4	2.95	3	4	1.13	-0.74	-0.45
Item 5	2.56	3	3	0.83	-1.40	1.97
Item 6	1.40	1	2	1.06	0.33	-0.52
Item 7	1.69	2	2	0.71	-0.45	1.00
Item 8	3.00	4	4	1.24	-1.04	0.02
Item 9	3.19	4	4	1.08	-1.28	0.93
Item 10	2.23	2	2	0.79	-0.24	0.25
Item 11	1.28	1	2	0.93	0.12	-0.80
Item 12	3.17	4	4	1.00	-1.04	0.45

• CASQ = Cancer Appetite and Symptom Questionnaire; SD = standard deviation.

None of the items showed severe deviation from normality, indicating appropriate psychometric sensitivity.

Figure 1 illustrates the path diagram of the CFA performed for the original CASQ structure and for the refined fit model to the sample of UK patients with cancer.

Figure 1
Factorial structure of the original model of the CASQ and of the refined fit model to the sample of United Kingdom patients with cancer. AIC = Akanke information criterion; AVE = average variance extracted; BCC = Browne-Cudeck criterion; BIC = Bayesian information criterion; CASQ = Cancer Appetite and Symptom Questionnaire; e = items with correlation between errors; CFA = confirmatory factor analysis; CFI = comparative fit index; CR = composite reliability; GFI = goodness of fit index; RMSEA = root mean square error of approximation.

We observed that in the original model, two items (6 and 12) had factor weights lower than 0.30. Thus, to improve the psychometric qualities of the CASQ for the UK sample, we decided to remove item 6 and included four correlations between errors (e1-e3; e4-e8; e7-e8; e10-e11). These decisions were made using modification indices. The decision to keep item 12 (λ < 0.30) was guided by its theoretical significance as an important manifestation of decreased appetite. This item evaluates pain, which is an aspect that should be considered when evaluating different clinical conditions including appetite in patients with cancer.

We observed a reduced convergent validity (AVE = 0.34) of models. The reliability was adequate (CR = 0.84, α = 0.84).

The only structural difference between the CASQ fit models to the UK and Brazilian samples is related to item 5 (number of meals per day), which remained in the UK sample but had to be excluded from the Brazilian sample so that the CASQ showed appropriate fit (CFA: λ = 0.34-0.70; χ²/df = 8.532, CFI = 0.94, GFI = 0.95, RMSEA = 0.08). ⁸ Thus, seeking the establishment of a common model between countries, Figure 2 shows the fit to the UK sample model without items 5 and 6.

Figure 2
Factorial structure of the CASQ without items 5 and 6 fit to the sample of United Kingdom cancer patients. Confirmatory factor analysis: λ= 0.28 to 0.82; χ²/df = 2.773, comparative fit index = 0.969, goodness of fit index = 0.968, root mean square error of approximation = 0.057, Akanke information criterion = 133.965, Browne-Cudeck criterion = 237.621, Bayesian information criterion = 134.938, average variance extracted = 0.35, composite reliability = 0.83, α = 0.84). e = items with correlation between errors.

We observed an appropriate fit of the reduced model to the data, with that structure being more simple (lowest AIC, BIC and BCC). It was further observed that the explained variance of this model was identical to that of the refined model shown in Figure 1 (refined model), which indicates that item 5 did not contribute significantly to the appetite and symptom construct in the UK sample.

When evaluating the factorial invariance of the CASQ between the UK and Brazilian samples, the model, while appropriately fitting the two samples, was not invariant (λ: ∆χ² = 64.008, p < 0.001; i: ∆χ² = 3515.047, p < 0.001; Res: ∆χ² = 4452.504, p < 0.001). This is observed by the differences between the explained variances (Brazil = 64%; UK = 88%), as also observed in the differences between factorial weights of the items in Figure 2 and in Spexoto et al. ⁸

To explore whether this finding was attributable to cultural differences between the countries or to differences between the samples, it was decided to also estimate the invariance between palliative and in-treatment samples from the two countries ( Table 3 ).

Table 3
CFA of the CASQ applied to the Brazilian and UK samples according to form of treatment

CFA	Brazil		United Kingdom
	Palliative care	In treatment	Palliative care	In treatment
λ	0.10-0.80	0.26-0.71	0.28-0.83	0.24-0.83
χ2/df	1.58	8.50	2.16	1.96
CFI	0.94	0.91	0.97	0.93
GFI	0.92	0.94	0.96	0.91
RMSEA	0.07	0.09	0.05	0.08
σ2 (%)	88	65	92	74

• CASQ = Cancer Appetite and Symptom Questionnaire; CFA = confirmatory factor analysis; CFI = comparative fit index; GFI = goodness of fit index; RMSEA = root mean square error of approximation.

The model was appropriately fit to the samples, but did not provide transnational invariance. The explained variance by the model was higher for the UK for patients in both the palliative and the in-treatment subgroups. The explained variance by the model was higher for both countries in the palliative treatment samples. The model was not invariant between countries for palliative treatment samples (λ:∆χ² = 28.210, p = 0.001; i: ∆χ² = 894.851, p < 0.001; Res: ∆χ² = 1433.827, p < 0.001) or for in-treatment samples (λ: ∆χ² = 23.254, p = 0.006; i: ∆χ² = 1348,530, p < 0.001; Res: ∆χ² = 2289,012, p < 0.001).

Discussion

This study complements the evidence presented by Halliday et al. ⁷ on the psychometric properties of the CASQ and its use as a tool to predict patients at high risk of future weight loss in order to target dietetic and anticachexia support in UK individuals with cancer.

Although the factorial model was not invariant, it was suitable for both countries and therefore can be considered in the present study samples as a valid method of evaluating appetite and symptoms in patients with cancer.

In the UK sample responses, to item 6 referring to snacking in addition to, or in replacement of, meals did not provide sufficient factorial weight to support its maintenance to capture the construct and thus it was removed from the instrument. This new model was simpler and showed better fit to the sample. This finding was replicated in the Brazilian sample, where item 6 was also removed due to the low factor weight. In order to complete the evaluation of transnational invariance, and for the model to be identical across the two countries, it was also necessary to remove item 5 from the model. Because items 5 and 6 refer to the frequency of meals and/or snacks during the day, their interpretation may be influenced by the individual’s perception of the meaning of these words.

The absence of model invariance across Brazilian and UK samples may be attributed to cultural differences that certainly influenced the process of building the construct held by the participants. It may be noted that the explained variance by the model applied to the UK sample was significantly higher than that observed in the Brazilian sample, suggesting that the appetite symptoms assessed presented a clearer theoretical value for the UK patients. Furthermore, the explained variance was also higher for samples that included participants undergoing palliative care. A possible explanation for this is related to how the initial content validation work to develop the CASQ was conducted, i.e., with an expert panel consisting of UK clinicians, caregivers and palliative care patients. It should be pointed out that the differences between samples ( Table 1 ) do not seem to be responsible for the lack of invariance; when comparing subsamples ( Table 3 ), the same differences held.

Analysis of the results should consider some study limitations, such as its cross-sectional design, which prevents the establishment of causality, and the differences in the characterization of the samples. Despite these limitations, this study screened a large sample and included individuals with different diagnoses, treatments, and clinical conditions. In addition, we sought to divide the sample into individuals under palliative and curative treatment in an attempt to homogenize the characteristics of the samples from both countries, so as to minimize the impact that these differences could exert on the evidence presented. The present study is, to the best of our knowledge, the first to evaluate the construct validity of the CASQ.

Given the significant impact that appetite and associated symptoms may have on the health status of individuals with cancer, findings suggest that the studies for different cultural contexts using the CASQ extend the knowledge of its psychometric properties. It should be noted that knowledge of these properties is essential for assessing the validity and reliability of the instrument and thus the information about the quality of data collected with it. ^9,10 Further studies with samples from different countries may increase the knowledge of professionals seeking the safe use of the CASQ directed at particular clinical settings.

Conclusion

The psychometric properties of the CASQ are sufficient to support its use in UK patients with cancer. However, the operationalization of the construct was different for the UK and Brazilian samples, being higher in the UK. There were also differences between patients receiving palliative care compared to those receiving anticancer treatment. These findings suggest that cultural, demographic and clinical specificities can impact on how patients with cancer report their appetite and symptoms. It is therefore recommended that the metric properties of the CASQ are properly evaluated before its use in different contexts and clinical settings.

Acknowledgements

This study was supported by Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP; grant #2011/22620-5) and Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES/PDSE; BEX 10091/14-9). Vanessa Halliday receives grants from the UK National Cancer Research Institute.

The authors thank the patients for their involvement in completing the CASQ.

References

• ¹
  Mercadante S, Casuccio A, Fulfaro F. The course of symptom frequency and intensity in advanced cancer patients followed at home. J Pain Symptom Manage. 2000;20:104-12.
• ²
  Bozzetti F, Mariani L. Defining and classifying cancer cachexia: a proposal by the SCRINIO Working Group. JPEN J Parenter Enteral Nutr. 2009;33:361-7.
• ³
  Bozzetti F. Nutritional support of the oncology patient. Crit Rev Oncol Hematol. 2013;87:172-200.
• ⁴
  Quinten C, Coens C, Mauer M, Comte S, Sprangers MA, Cleeland C, et al. Baseline quality of life as a prognostic indicator of survival: a meta-analysis of individual patient data from EORTC clinical trials. Lancet Oncol. 2009;10:865-71.
• ⁵
  Sood R, Jatoi A. Integrative nutritional approaches to loss of weight and appetite in patients with advanced cancer. Eur J Integr Med. 2011;3:e233-6.
• ⁶
  Wilson MM, Thomas DR, Rubenstein LZ, Chibnall JT, Anderson S, Baxi A, et al. Appetite assessment: simple appetite questionnaire predicts weight loss in community-dwelling adults and nursing home residents. Am J Clin Nutr. 2005;82:1074-81.
• ⁷
  Halliday V, Porock D, Arthur A, Manderson C, Wilcock A. Development and testing of a cancer appetite and symptom questionnaire. J Hum Nutr Diet. 2012;25:217-24.
• ⁸
  Spexoto MC, Serrano SV, Halliday V, Maroco J, Campos JA. Cancer Appetite and Symptom Questionnaire (CASQ) for Brazilian Patients: cross-cultural adaptation and validation study. PLoS One. 2016;11:e0156288.
• ⁹
  Campos J, Bonafé F, Dovigo L, Maroco J. Psychometric assessment of the attitudes towards statistics scale. Rev Bras Biom. 2013;31:327-37.
• ¹⁰
  Maroco J, Maroco AL, Campos JADB. Student’s academic efficacy or inefficacy? An example on how to evaluate the psychometric properties of a measuring instrument and evaluate the effects of item wording. Open J Stat. 2014;4:484-93.
• ¹¹
  Hair JF, Black WC, Babin B, Anderson RE, Tatham RL. Multivariate data analysis. 6th ed. Upper Saddle River: Prentice Hall; 2009.
• ¹²
  Percival C, Hussain A, Zadora-Chrzastowska S, White G, Maddocks M, Wilcock A. Providing nutritional support to patients with thoracic cancer: findings of a dedicated rehabilitation service. Respir Med. 2013;107:753-61.
• ¹³
  Maroco J. Análise de equações estruturais: fundamentos teóricos, softwares e aplicações. 2nd ed. Lisboa: Report Number; 2014.
• ¹⁴
  Fornell C, Larcker DF. Evaluating structural equation models with unobservable variables and measurement error. J Mark Res. 1981;18:39-50.
• ¹⁵
  Kaplan D. Structural equation modeling: foundations and extensions. 2nd ed. Thousand Oaks: Sage; 2000.

Publication Dates

• Publication in this collection
  14 May 2018
• Date of issue
  Apr-Jun 2018

History

• Received
  02 Aug 2017
• Accepted
  03 Dec 2017