Evaluation of inter-rater reliability of subjective and objective criteria for diagnosis of lymphedema in upper and lower limbs

Background: The diagnosis of lymphedema can be obtained objectively by measurement methods, and also by subjective methods, based on the patient’s complaint. Objective: To evaluate inter-rater reliability of objective and subjective criteria used for diagnosis of lymphedema and to propose a lymphedema cut-off for differences in volume between affected and control limbs. Methods: We studied 84 patients who had undergone lymphadenectomy for treatment of cutaneous melanoma. Physical measures were obtained by manual perimetry (MP). The subjective criteria analyzed were clinical diagnosis of lymphedema in patients’ medical records and self-report of feelings of heaviness and/or increase in volume in the affected limb. Results: For upper limbs, the subjective criteria clinical observation (k 0.754, P<0.001) and heaviness and swelling (k 0.689, P<0.001) both exhibited strong agreement with MP results and there was moderate agreement between MP results and swelling (k 0.483 P<0.001), heaviness (k 0.576, P<0.001) and heaviness or swelling (k 0.412, P=0.001). For lower limbs there was moderate agreement between MP results and clinical observation (k 0.423, P=0.003) and regular agreement between MP and self-report of swelling (k 0.383, P=0.003). Cut-off values for diagnosing lymphedema were defined as a 9.7% difference between an affected upper limb and control upper limb and a 5.7% difference between lower limbs. Conclusion: Manual perimetry, medical criteria, and self-report of heaviness and/or swelling exhibited better agreement for upper limbs than for lower limbs for diagnosis of lymphedema.


INTRODUCTION
Lymphadenectomy conducted to treat cutaneous melanoma causes lymphedema. Other risk factors for lymphedema are melanoma thickness >4 mm, infection and graft reconstruction, and a combination of these risk factors increases the chances of developing this chronic condition. 1 Lymphedema can be diagnosed using several different objective methods, including manual perimetry (MP), water displacement, tonometry, optoelectronic volumetry and bioimpedance. 2 However many studies have diagnosed lymphedema subjectively on the basis of patients' responses to questions about their symptoms, such as heaviness and/or swelling in the limb. [3][4][5] Several prospective and retrospective studies 3,4,6-10 have diagnosed upper and lower limb lymphedema secondary to treatment of melanoma using combinations of objective or subjective methods, for example, MP and optoelectronic volumetry; patient history and physical examination; self-report and medical records (Table 1).
Manual perimetry offers the advantages of low cost, requiring only a tape measure, and ease of use in clinical practice. It is a simple method that can be used regardless of skin condition and requires minimal technology or training. 11 Circumference measurements are taken at 7 or 10 cm intervals and then it is possible to calculate limb volume from the sum of each truncated cone, [12][13][14] using free online calculators such as www.armvolume.com and www. legvolume.com. Distances between measurements vary from 4 to 15 cm in different studies. [12][13][14][15][16] Studies comparing limb volume measurements calculated from water displacement with the results of geometric formulas using input values obtained by MP show excellent correlation, indicating that they are equally valid for diagnosis of lymphedema. 17,18 The objective of this study was to examine the inter-rater reliability of objective and subjective criteria used for diagnosis of lymphedema. It is also interesting to propose cut-offs for differences in volume between affected and control limbs that could be used to diagnose lymphedema.

METHODS
This study enrolled patients who underwent lymphadenectomy from 1990 to 2008 at our institution. The exclusion criteria were patients with limb amputation or bilateral lymph node dissection and patients who refused to participate. All patients were requested to have an interview and to be examined. During the period, 364 inguinal, ilioinguinal and axillary lymphadenectomies were conducted on patients with melanoma. From these 364 procedures, 186 patients had died from diseases or other causes during follow-up, 63 could not be located, five could not be evaluated because they were bedridden, 17 were excluded because of limb amputation, and seven were excluded because of bilateral dissection. Only two patients refused to participate in the study and so the final sample included 84 patients who had been diagnosed with cutaneous melanoma and had undergone axillary, groin, or ilioinguinal lymph node dissection with a minimum of six months' follow-up. The project was approved in advance by the Research Ethics Committee at the A.C.Camargo Cancer Center.
Manual perimetry was performed using a regular tape measure. For upper limbs, measurements were taken at 7 cm intervals; 7 and 14 cm above the interarticular line through the elbow and at 7, 14 and 21 cm below the line. For lower limbs, measurements were taken every 10 cm from the sole up to the seventh measurement. Measurements for all patients were obtained by a single researcher to prevent differences caused by varying tension in the tape. These data were fed into the truncated cone formula: where: V = volume of the final segment of the limb, C1 and C2 = circumference measured between the points, and h = distance between the circles (C1 and C2 in each segment) in centimeters. Differences between limb volumes measured by MP were calculated. Lymphedema was defined as a difference greater than 10% between upper limb volumes 12,19 or greater than 6.5% between lower limb volumes. 20,21 Lymphedema defined by MP was also compared to lymphedema diagnosed subjectively in physicians' reports (presence of lymphedema in the medical record, when there was a considerable visual difference between limbs) and patients' complaints (self-report of heaviness and/or swelling in the limb at the time of physical assessment, where the patient notices that his/her shirt sleeve or pants are tighter in the ipsilateral lymphadenectomy limb).
Inter-rater agreement between MP and patient/ medical criteria for diagnosing lymphedema was calculated using the Kappa (k) index. Receiver operator characteristics (ROC) curve analysis was used to establish cut-off values for the difference between limb volumes measured by MP according to the patient and medical criteria. In all statistical tests, significance was accepted at the 5% level. The Statistical Package for the Social Sciences (SPSS  ) version 15.0 (Chicago, IL) was used for statistical analyses.

RESULTS
Forty-eight patients were women (57.1%). The average age of patients at surgery was 47.2 years (sd: 16.7 years), ranging from five to 80 years and on the day of assessment, 52.5 years (sd: 16 years), ranging from 10 to 81 years. There were only three people (3.6%) under 18 years old.
Eighty-four patients were evaluated, 40 (47.6%) had had axillary lymph node dissection, 21 (25%) inguinal and 23 (27.4%) had undergone ilioinguinal lymphadenectomy. The mean time elapsed since lymphadenectomy was 62.5 months (sd: 56.1 months; median: 44 months), ranging from six months thru 17.6 years. The prevalence rates of lymphedema (according to MP) were 17.5% in upper limbs and 59.1% in lower limbs, while prevalence rates of lymphedema according to subjective methods (self-report of heaviness and/or swelling and medical record) were 32.5% in upper and 66% in lower limbs. Table 2 shows prevalence rates of lymphedema according to type of lymphadenectomy. Lymphedema was more common after ilioinguinal than after axillary or inguinal lymphadenectomy. Self-report of swelling in the affected limb was the most common complaint, mainly in ilioinguinal lymphadenectomy patients (95.7%). Some patients without lymphedema (according to MP) complained of heaviness and/ or swelling in the limb, irrespective of type of lymphadenectomy.
Comparison of the different diagnostic methods for lymphedema of upper limbs revealed significant and moderate agreement between MP and subjective patient criteria as follows: self-report of swelling (k=0.483; P<0.001), heaviness (k=0.576; P<0.001) and heaviness or swelling (k=0.412; P=0.001). Selfreport of heaviness and swelling (k=0.689; P<0.001) and medical criteria (k=0.754; P<0.001) exhibited strong agreement. When results for lymphedema of lower limbs were compared, significant and regular agreement was detected between MP result and swelling (k=0.383; P=0.003), and there was moderate agreement with medical records (k=0.423; P=0.003) ( Table 3).
Cut-off values for differences between limb volumes measured by MP calculated from patient and medical criteria for defining lymphedema were as follows: a 9.7% difference between an affected upper limb and control upper limb, and a 5.7% difference for lower limbs. The subjective criteria were also analyzed individually according to MP results, as shown in Table 4. Heaviness, swelling and heaviness and medical records all resulted in the same cut-off (9.6%) for upper limb, while swelling and swelling or heaviness gave a cutoff of 4.7%. The cut-off points for lower limbs were more divergent: 9.5% for medical records and swelling or heaviness, 13.8% for heaviness, 4.4% for swelling and 4.8% for swelling and heaviness.

DISCUSSION
There is a relatively low number of patients with melanoma, compared to other tumors, and because of this most studies of melanoma have restricted numbers of participants and this is also a limitation of this study. It would have been interesting if these patients could have been studied prospectively.
Some studies have correlated the incidence of morbidity from lymphedema with axillary, inguinal or ilioinguinal lymphadenectomies used to treat cutaneous melanoma. In most cases, studies that mention lymphedema as the primary complication define it on the basis of subjective methods only. It is of interest to investigate lymphedema based on numerical values, using formulas or equipment to provide the data. A comparison of MP with medical diagnosis of lymphedema and self-report measures is interesting because the results can be used to compare different diagnostic techniques.
Kretschmer et al. 3 and Sabel et al. 4 analyzed lymphedema using self-report of swelling and heaviness of affected vs. control limbs. The rate of lymphedema was 19% (mentioned only in the first study) in upper limbs and 63% and 30%, respectively, for lower limbs. In contrast, the values obtained in our analysis were higher at 34% for upper limbs and 76% for lower limbs. Both of the studies cited above had larger numbers of participants than this one.
McLaughlin et al. 22 reported a significant discrepancy between self-report and measured lymphedema. Hayes et al. 23 concluded that self-report measures offered good sensitivity for diagnosis real lymphedemas, but bad specificity for patients who did not have lymphedema according to clinical examination. In our study, some patients without lymphedema, according to MP, complained of swelling and/or heaviness, while most patients who did not have heaviness and/or swelling did not have lymphedema.
Armer and Stewart 24 studied breast cancer survivors, comparing four diagnostic criteria: optoelectronic volumetry using a Perometer 350 S (lymphedema was considered as 200 ml and 10% changes in limb volume); MP (2 cm between affected and control upper limb) and self-report of heaviness or swelling during the assessment and/or in the past. It was concluded that there is no gold standard, but that the best criteria was using perometry with 10% limb volume changes. Circumferences using 2 cm was the worse definition. The criteria self-report of heaviness and swelling were better than perometry with 200 ml.
Recently, another study also compared four diagnostic methods in 295 patients post breast cancer to observe the effect of weight lifting on lymphedema. They defined lymphedema as interlimb change ≥10% for water displacement and MP results and also used bioimpedance and self-report.

Diagnosis of lymphedema
None of these methods were considered a gold standard. It was concluded that it is important to use multiple methods to evaluate patients with axillary lymphadenectomy. 25 In another study that started with 511 women and completed with 176, the prevalence of lymphedema following cancer breast treatment ranged from 0.6 to 27.8%. It was observed that prevalence of lymphedema was higher when determined by selfreport (arm swelling) and bioimpedance than when sum of arm circumferences was used. The cut-offs for lymphedema according to sum of circumferences were greater than 5 cm or greater than 10% (as in our paper). Lymphedema was most prevalent according to self-report measures and lowest according to circumferences. Bioimpedance was considered the best method and the use of circumferences was questioned because it exhibited limitations. 23 Smoot et al. 26 analyzed 144 women with breast cancer using bioimpedance, truncated cone MP and a self-report questionnaire and concluded that patients with lymphedema must be evaluated using both selfreport and objective methods.
Early lymphedemas can be detected when patients notice changes causing sensations such as heaviness and swelling in the upper limb. The ideal would be to relate limb volume measurements to symptoms. 27 Patients who did not have lymphedema according to measurements did report swelling and/or heaviness in the affected limb and this should lead us to monitor these individuals closely, since they could have a greater chance of developing lymphedema. Tiwari et al. 28 reported that the first symptom of lymphedema is complaining of a feeling of heaviness in the limb, especially at the end of the day and on days with higher temperatures.
Most studies of lymphedema are with breast cancer patients and the cut-off value for diagnosing lymphedema is >10% interlimb change for upper limbs. [23][24][25] The problem is to define a cut-off value for lower limbs. Using optoelectronic volumetry, Spillane et al. 8 studied 66 patients who had undergone inguinal or ilioinguinal dissection and considered lymphedema to be a volume difference ≥15% between affected and control lower limbs using Perometer or ≥7% difference using sum of circumferences. Katz et al. 29 considered lymphedema to be a volume difference between limbs greater than 6% using optoelectronic volumetry.
The cut-off points for diagnosis of lymphedema identified in this study were a 9.7% difference in volume in the affected upper limb vs. the control upper limb and a 5.7% difference in volume between lower limbs. These findings were similar to results from several other studies 19,23-25 that defined lymphedema as a difference of >10% in upper limbs, and a mean of 6.5% difference for lower limbs. 8,21,29 In the analysis of results for upper limbs, selfreport of heaviness in the affected limb, swelling and heaviness and diagnosis of lymphedema by a physician all had very similar cut-off values (9.6%). For lower limbs, subjective diagnosis of lymphedema was varied. In severe lymphedema, simple observation and palpation of the limb is sufficient for diagnosis, but mild or moderate lymphedema can go unnoticed. For this reason, it is essential to ask patients about their limb complaints (swelling and heaviness). It is also essential for health professionals to take accurate, focused medical histories with regards to the affected limb and to use MP or other objective methods to arrive at more precise diagnoses. Manual perimetry is also useful for observing a limb's progress, to monitor whether it remains normal or develops lymphedema and to follow the response to physical therapy.

CONCLUSION
Manual perimetry, medical records and selfreport of heaviness and swelling in the affected limb exhibited better agreement for upper limbs than for lower limbs when used to diagnose lymphedema. Cut-off values for diagnosing lymphedema were defined as a 9.7% difference between an affected upper limb and a control upper limb, and a 5.7% difference between lower limbs. More studies must be conducted using different objective methods and correlating the results with subjective criteria, particularly for lower limbs.