Comparison among ACR1997, SLICC and the new EULAR/ACR classification criteria in childhood-onset systemic lupus erythematosus

Abstract

Background:

To date there are no specific classification criteria for childhood-onset systemic lupus erythematosus (cSLE). This study aims to compare the performance among the American College of Rheumatology (ACR) 1997, the Systemic Lupus International Collaborating Clinics criteria (SLICC) and the new European League Against Rheumatism (EULAR)/ACR criteria, in a cSLE cohort.

Methods:

We conducted a medical chart review study of cSLE cases and controls with defined rheumatic diseases, both ANA positive, to establish each ACR1997, SLICC and EULAR/ACR criterion fulfilled, at first visit and 1-year-follow-up.

Results:

Study population included 122 cSLE cases and 89 controls. At first visit, SLICC criteria had higher sensitivity than ACR 1997 (89.3% versus 70.5%, p < 0.001), but similar specificity (80.9% versus 83.2%, p = 0.791), however performance was not statistically different at 1-year-follow-up. SLICC better scored in specificity compared to EULAR/ACR score ≥ 10 at first visit (80.9% versus 67.4%, p = 0.008) and at 1-year (76.4% versus 58.4%, p = 0.001), although sensitivities were similar. EULAR/ACR criteria score ≥ 10 exhibited higher sensitivity than ACR 1997 (87.7% versus 70.5%, p < 0.001) at first visit, but comparable at 1-year, whereas specificity was lower at first visit (67.4% versus 83.2%, p = 0.004) and 1-year (58.4% versus 76.4%, p = 0.002). A EULAR/ACR score ≥ 13 against a score ≥ 10, resulted in higher specificity, positive predictive value, and cut-off point accuracy. Compared to SLICC, a EULAR/ACR score ≥ 13 resulted in lower sensitivity at first visit (76.2% versus 89.3%, p < 0.001) and 1-year (91% versus 97.5%, p = 0.008), but similar specificities at both assessments. When compared to ACR 1997, a EULAR/ACR total score ≥ 13, resulted in no differences in sensitivity and specificity at both observation periods.

Conclusions:

In this cSLE population, SLICC criteria better scored at first visit and 1-year-follow-up. The adoption of a EULAR/ACR total score ≥ 13 in this study, against the initially proposed ≥10 score, was most appropriate to classify cSLE. Further studies are necessary to address if SLICC criteria might allow fulfillment of cSLE classification earlier in disease course and may be more inclusive of cSLE subjects for clinical studies.

Keywords:
Systemic lupus erythematosus; Childhood; Adolescence; Classification criteria

Background

Systemic lupus erythematosus (SLE) is a chronic autoimmune disease, with a broad spectrum of clinical patterns. SLE affects women predominantly at reproductive age but may present at any age [¹]. Childhood-onset SLE (cSLE) represents approximately 20% of SLE cases [²] and displays a higher frequency of atypical manifestations, more severe presentation and course, higher rates of disease activity and cumulative damage, than that reported for adult-onset disease [³, ⁴].

As the purpose of classification criteria is to identify a well-defined patient population suitable for research, specificity thus generally outweighs sensitivity in determining classification performance.

The SLE classification criteria set most commonly used is the one established by the American College of Rheumatology (ACR) [⁵]. Besides the development in adult-onset SLE and scarce validation in cSLE, concerns arose about the redundancy of photosensitivity with skin rashes, non inclusion of several clinically relevant integument and nervous system lupus manifestations, as well as hypocomplementemia, inadequate quantification of urine protein by dipstick, and, classification as SLE for patients without positive autoantibodies [⁶].

Alternative methods for SLE classification have been developed predominantly in adults, such as the SLICC (Systemic Lupus International Collaborating Clinics) criteria [⁷] and the new EULAR/ACR criteria [⁸-¹²].

The main changes proposed in SLICC criteria were the expansion of cutaneous and neurological criteria, allocation of cytopenias and autoantibodies in individual criterion, inclusion of alopecia and hypocomplementemia, and the classification for patients with only documented lupus nephritis with ANA or anti-dsDNA [⁷]. SLICC criteria yielded higher sensitivity (97% versus 83%) but lower specificity (84% versus 96%) than ACR 1997 criteria in the original validation set [⁷]. In subsequent studies in adult and cSLE, SLICC higher sensitivity was also found, especially for early SLE, however, results were conflicting regarding specificity [¹³-²⁰]. The SLICC criteria have also been criticized because they were derived comparing the expert's decision (“gold standard”) with a standardized group of manifestations [⁶]. Moreover, as SLICC criteria emphasize immunological and hematological events, it might be possible that subjects classified through SLICC criteria may exhibit less clinically significant multisystem involvement compared with subjects classified through ACR criteria [⁶].

In 2017, the EULAR and ACR joined in a four-phase project to develop more sensitive (especially for initial classification) and more specific SLE classification criteria [⁸-¹²]. The first phase of this project was designed to gather potential candidate items broadly, through an SLE expert Delphi exercise and an international early SLE cohort study. The second phase consisted of item reduction by nominal group technique. The third phase was for item definition (literature based) and weighting (multiparameter decision analysis) and the fourth phase for item testing and validation against ACR 1997 and SLICC [¹⁰]. These criteria rely on a scoring system for clinical and laboratory domains [²¹], and a positivity of antinuclear antibody (ANA) at a titer 1:80 or higher by immunofluorescence (IFA) as an entry criterion [²²]. The patient is classified as SLE if the total score is equal to or greater than 10 [¹⁰, ¹¹]. EULAR/ACR criteria were tested, simplified and validated in a large (n = 2.218) international cohort. Performance characteristics found a sensitivity similar to the SLICC criteria (98% versus 95% for SLICC and 85% for ACR 1997) while maintaining the specificity of the ACR 1997 criteria (97% versus 95% for ACR 1997 and 90% for SLICC) [¹⁰]. Limitations indicated by the authors were the possible misclassification of patients with overlapping syndromes and the non-inclusion of new biomarkers [¹⁰, ¹⁹]. Other authors pointed out that the lack of extensive data on the longitudinal expression of ANA could affect the application of classification criteria in which ANA expression is the entry point [²³].

This study aims to compare the performance of ACR 1997, SLICC criteria and the new EULAR/ACR criteria, to identify patients with cSLE at first visit and 1-year-follow-up.

Methods

Inclusion criteria

Children and adolescents, with cSLE (cases) or other defined rheumatic diseases (controls), with ANA reactivity at ≥1:80 serum dilution, followed-up at the Pediatric Rheumatology Unit of our University Hospital, from 2000 to 2017, were consecutively selected, from the number of patients evaluated in the clinic during the inclusion period.

To be included, cases and controls patients needed to have a well-established clinical diagnosis, performed and confirmed by three highly experienced pediatric rheumatologists, with over than 20 years experience in pediatric rheumatology and cSLE, of the medical staff of the outpatient clinic. All baseline and evolutionary information (physical examination, laboratory parameters, and imaging), was routinely discussed and re-evaluated at each visit, by attending pediatric rheumatologists, which established the diagnosis based on continuous follow-up of all patients and total agreement about diagnosis, supported by internationally accepted criteria [²⁴-²⁹].

Exclusion criteria

Patients with overlapping syndromes or undifferentiated disease and those patients followed-up for less than 1 year were excluded.

Data collection

We performed a medical chart review for all eligible patients, and information collected in a standardized file. Data collection was retrospective, extracted by two of the authors and reviewed by the other three authors before classification sets scoring. Discrepancies were solved by team discussion. Finally, all patient's files (from cSLE cases and controls) were rated for each ACR 1997, SLICC and EULAR/ACR criterion that was or was not met, as laid out by the respective classification rule, at first visit, and at 1-year-follow-up. Cases and controls were classified as SLE if met ≥4 criteria for ACR 1997, ≥ 4 criteria or documented lupus nephritis with ANA, anti-dsDNA or both for SLICC and ≥ 10 or ≥ 13 total score for EULAR/ACR.

Baseline/first visit data were those obtained from the clinical history, physical examination and laboratory tests requested by attending pediatric rheumatologists at first visit. The immunologic assessments evaluated for cSLE and control patients were antinuclear antibody (ANA), anti-dsDNA, anti-Sm, anticardiolipin IgM and IgG, lupus anticoagulant, anti- β2-glycoprotein-I IgM and IgG (for patients who started follow-up after the year of 2012), direct Coombs test, levels of complement proteins C3 and C4, and VDRL (Venereal Disease Research Laboratory).

Criteria definitions

Definitions for each clinical or laboratory criterion were those provided by the respective criteria set. 1) ANA by indirect immunofluorescence, on human cell epithelioma (HEp-2) cells substrate, defined as positive if staining reactivity at ≥1:80; 2) Anti-dsDNA by indirect immunofluorescence, on Crithidia lucilae substrate, described as positive if staining reactivity at > 1:10 serum dilution; 3) Anti-Sm by enzyme-linked immunosorbent assay (ELISA), considered positive if above kit manufacturer cut-off value. 4) Anticardiolipin (aCL) IgM and IgG by ELISA, a cut-off value of 20 MPL or GPL for ACR1997 and SLICC criteria, and a cut-off value of 40 MPL or GPL for EULAR/ACR criteria set.

EULAR/ACR criteria: A positivity of antinuclear antibody (ANA) at a titer 1:80 or higher by immunofluorescence (IFA) is required as an entry criterion [²²]. These criteria rely on weighted additive criteria divided into seven clinical domains and three immunological domains, for which attribution to SLE is critical [²¹]. For each domain, only the individual criterion of highest value is considered for the total score [¹⁰, ¹¹]. Clinical domains include: 1) unexplained fever; 2) arthritis; 3) serositis (pleural or pericardial effusion, acute pericarditis); 4) mucocutaneous (acute cutaneous lupus, sub-acute/discoid lupus, alopecia, oral ulcers); 5) central nervous system involvement (seizures, psychosis and delirium); 6) hematological involvement (autoimmune hemolytic anemia, thrombocytopenia and leukopenia); 7) nephritis (proteinuria > 0.5 g/d, nephritis class III/IV, nephritis class II/V). Lupus nephritis class II/IV over class II/V gain the highest weight. Immunological domains consist of 1) autoantibodies (anti-dsDNA, anti-Sm); 2) low complement (both C3 and C4 OR only C3 or C4); 3) Antiphospholipid antibodies (anticardiolipin IgG ≥ 40 GPL or anti-β2 glycoprotein I IgG ≥ 40 GPL or lupus anticoagulant). The patient is classified as SLE if the total score is equal to or greater than 10, in the presence of at least one clinical criterion, and each criterion may occur serially or simultaneously and on at least one occasion [¹⁰, ¹¹].

Statistical analysis

The Statistical Package for the Social Sciences (SPSS) 20.0 for Windows (IBM, Armonk, NY, USA) was used. Data were shown in median (range, IQR) for continuous variables, and further comparisons between groups were conducted using Student's t-test. Differences between proportions or categorical variables were analyzed by Fisher exact test. We performed the calculation of two types of accuracy: global accuracy and cut-off point accuracy [³⁰]. The global accuracy was calculated for all three criteria sets, through the area under the ROC curve (AUC), to estimate how high is the discriminative power of each criteria set. However, as the AUC tell us nothing about individual cut-off points, we calculated the cut-off point accuracy for ACR 1997 ≥ 4 criteria, for SLICC ≥4 criteria and EULAR/ACR total score (total score ≥ 10, ≥11 and ≥ 13), through two-by-two contingency tables. McNemar's test was applied to assess differences in sensitivity and specificity between ACR1997, SLICC and EULAR/ACR classification sets. A p < 0.05 value was regarded as significant, and all analyses were two-tailed.

Ethics

The local Research Ethics Committee approved the study protocol before study commencement, under Number 2.421.080, on December 7th, 2017.

Results

Sample description

133 cSLE and 96 controls fulfilled inclusion criteria, but there were 11 (9.0%) losses for cases group and 7 (8.2%) for control group due to missing information about some variables in medical charts. Study population included 122 cases with a well-established clinical diagnosis of cSLE (82.8% female, median onset age of 10.32 years (2.8-17.1) and median follow-up time of 6.0 years) and 89 controls (75.3% female, median onset age of 9.0 years and median follow-up time of 6.0 years), see Table 1.

Table 1
General characteristics of case and control groups

	Cases (N = 122)	Controls (N = 89)	P value
Sex ratio (female: male)	101:21	67:22	0.226
Median onset age, years	10.32	9.00	< 0.001
(range)	(2.8-17.1)	(1.1-15.8)
Interquartile range (years)	3.2	6.3
Median age at diagnosis, years	10.63	9.5	0.024
(range)	(4.1-17.3)	(1.9-17.8)
Interquartile range (years)	3.0	7.0
Median time to diagnosis, months	3.00	9.00	< 0.001
(range)	0-60	1-68
Interquartile range (years)	4.0	18.5
Median follow-up time, years	6.00	6.00	0.91
(range)	1-13	2-16
Interquartile range (years)	4.0	4.0

• Cases were patients with a well-established clinical diagnosis of childhood-onset SLE (cSLE). Controls had systemic-onset juvenile idiopathic arthritis, JDM juvenile dermatomyositis, JSS juvenile systemic sclerosis, MCTD mixed connective tissue disease, SS Sjogren syndrome, APS primary antiphospholipid syndrome, or primary vasculitis (Behçet, polyarteritis nodosa, Takayasu's arteritis and granulomatosis with polyangiitis)

Controls had the following diagnoses: 8 systemic-onset juvenile idiopathic arthritis (SoJIA), 34 juvenile dermatomyositis (JDM), 10 juvenile systemic sclerosis (JSS), 14 mixed connective tissue disease (MCTD), 12 Sjogren syndrome (SS), 3 primary antiphospholipid syndromes (APS), 8 primary vasculitides.

ACR 1997 criteria

The most commonly observed ACR criteria in cSLE cases, both at the first visit and at 1-year-follow-up, respectively, were: arthritis (77 and 86.1%), immunologic (59 and 76.2%), hematologic (59 and 73.8%) and malar rash (36.9 and 52.5%). Arthritis (64 and 70.8%), malar rash (19.1 and 29.2%), and hematological (18 and 21.3%) were the most observed criteria in the control group.

At first visit, the mean number of ACR criteria was 4.54 ± 0.16 for cases and 2.52 ± 0.12 for controls (p < 0.001). At 1-year-follow-up, this average was 5.71 ± 0.14 for cases and 2.82 ± 0.12 for controls (p < 0.001).

Fifteen controls were misclassified as JSLE at first visit: 1 SoJIA, 8 JDM, 1 JSS, 4 MCTD, and 1 SS. Six additional controls were misclassified at 1-year-follow-up: 1 JDM and 5 MTCD.

SLICC criteria

The most frequent SLICC criteria in cSLE, both at first visit and at 1-year-follow-up, were, respectively: arthritis (78.7 and 87.7%), hypocomplementemia (56.6 and 60.7%), acute cutaneous lupus (49.2 and 65.6%), alopecia (42.6 and 44.3%) and anti-dsDNA (37.7 and 53.3%). At first visit, cSLE cases fulfilled a mean of 6.07 ± 0.21 and controls 2.69 ± 0.13 SLICC criteria (p < 0.001). At 1-year-follow-up, cases had a mean of 7.42 ± 0.20 and controls 3.00 ± 0.14 (p = 0.012) SLICC criteria.

Seventeen controls were misclassified as cSLE at first visit: one SoJIA, 2 JDM, 2 JSS, 9 MCTD, 1 primary APS, one SS, and one primary vasculitis. Four additional controls were misclassified at 1-year-follow-up: 2 JDM and 2 MCTD.

Table 2 presents the prevalence of SLICC criteria in cSLE cases and controls at baseline/first visit and 1-year-follow-up.

Table 2
Prevalence of clinical and Immunological criterion of Systemic Lupus International Collaborating Clinics (SLICC) Classification Criteria in 122 cSLE cases and 89 controls, at baseline/first visit and 1-year-follow-up

Criterion		Cases Baseline/First visit n (%)	Controls Baseline/First visit n (%)	P value	Cases 1-year follow-up n (%)	Controls 1-year follow-up n (%)	P value
Acute cutaneous lupus		60 (49.2)	22 (24.7)	< 0.001	80 (65.6)	30 (33.7)	< 0.001
	Malar rash	45 (36.9)	17 (19.1)	0.006	64 (52.5)	26 (29.2)	0.001
	Photosensitive lupus rash	29 (23.8)	14 (15.7)	0.169	34 (27.9)	16 (18.0)	0.104
	Subacute cutaneous lupus	18 (14.8)	1 (1.1)	< 0.001	21 (17.2)	1 (1.1)	< 0.001
Chronic cutaneous lupus		13 (10.7)	1 (1.1)	0.005	16 (13.1)	1 (1.1)	0.001
	Discoid rash	13 (10.7)	1 (1.1)	0.005	15 (12.3)	1 (1.1)	0.003
Oral ulcers		44 (36.1)	11 (12.4)	< 0.001	64 (52.5)	12 (13.5)	< 0.001
Alopecia		52 (42.6)	12 (6.8)	< 0.001	54 (44.3)	14 (9.1)	< 0.001
Synovitis		96 (78.7)	59 (66.3)	0.058	107 (87.7)	65 (73.0)	0.011
Serositis		30 (24.6)	3 (3.4)	< 0.001	41 (33.6)	4 (4.5)	< 0.001
	Pleuritis	22 (18.0)	2 (2.3)	< 0.001	32 (26.2)	3 (3.4)	< 0.001
	Pericarditis	8 (6.6)	0	0.022	9 (7.4)	2 (2.3)	0.124
Renal disorder		33 (27.0)	1 (1.1)	< 0.001	49 (40.2)	2 (2.3)	< 0.001
	Proteinuria	33 (27.0)	1 (1.1)	< 0.001	49 (40.2)	2 (2.3)	< 0.001
	Red blood cell casts	10 (8.2)	0	< 0.001	15 (12.3)	1 (1.1)	< 0.001
Neuropsychiatric		10 (8.2)	1 (1.1)	0.027	23 (18.9)	1 (1.1)	< 0.001
	Seizures	5 (4.1)	0	0.075	17 (13.9)	0	< 0.001
	Psychosis	2 (1.6)	0	0.510	6 (4.9)	0	0.041
	Myelitis	1 (0.8)	0	0.999	3 (2.5)	0	0.265
	Confusional state	1 (0.8)	0	0.999	1 (0.8)	0	0.999
Hemolytic anemia		44 (36.1)	4 (4.5)	< 0.001	61 (50.0)	4 (4.5)	< 0.001
Leukopenia/lymphopenia		39 (32.0)	(10.1)	< 0.001	53 (43.4)	11 (12.4)	< 0.001
Thrombocytopenia		18 (14.8)	4 (4.5)	0.021	20 (16.4)	4 (4.5)	0.008
Anti-dsDNA		46 (37.7)	4 (4.5)	< 0.001	65 (53.3)	5 (5.6)	< 0.001
Anti-Sm		29 (23.8)	5 (5.6)	< 0.001	38 (31.1)	5 (5.6)	< 0.001
Antiphospholipid		30 (24.6)	5 (5.6)	< 0.001	38 (31.1)	4 (4.5)	< 0.001
	Anticardiolipin IgM	11 (9.0)	2 (2.3)	0.047	12 (9.8)	2 (2.3)	0.046
	Anticardiolipin IgG	23 (18.9)	2 (2.3)	< 0.001	27 (22.1)	3 (2.5)	< 0.001
	Lupus anticoagulant	6 (4.9)	5 (5.6)	0.999	8 (6.6)	5 (5.6)	0.999
	Anti-β2 GPI I IgM	3/61 (4.9)	1/42 (2.4)	0.644	3/61(4.9)	1/42(2.4)	0.644
	Anti-β2 GPI I IgG	3/61 (4.9)	2/42 (4.9)	0.999	3/61(4.9)	2/42(4.9)	0.999
	False-positive VDRL	4 (3.3)	1 (1.1)	0.4	4 (3.3)	1 (1.1)	0.4
Low complement		69 (56.6)	4 (4.5)	< 0.001	74 (60.7)	5 (5.6)	< 0.001
Direct Coombs test		34 (27.9)	6 (6.7)	< 0.001	38 (31.1)	6 (6.7)	< 0.001

• The Pearson chi-square test analyzed differences between proportions, and the difference was regarded as statistically significant when p < 0.05 No patients had toxic epidermal necrolysis, hypertrophic lupus, mucosal lupus, lupus tumidus, and chilblains lupus. One cSLE case had bullous lupus, another one cSLE had panniculitis, and one control had mononeuritis multiplex at 1-year-follow-up. Anti-β2 glycoprotein-I IgM and IgG were tested in 61/122 (50%) cSLE cases and 42/89 (47.2%) controls, at baseline/first visit and 1-year-follow-up

EULAR/ACR criteria

To better evaluate the use of EULAR/ACR criteria in our cSLE population, we searched through ROC curve analysis, the total score cut-off that might optimize sensitivity and specificity, and we noticed that a total score ≥ 13, against the initially proposed ≥10 score, was more suitable for our cSLE patients. Besides, we also decided to perform a sensitivity analysis for different cut-off points of EULAR/ACR total score.

At first visit, if a EULAR/ACR total score ≥ 13 is adopted, a lower sensitivity (76.2% versus 87.7% for score ≥ 10, p < 0.0001) but a higher specificity (87.6% versus 67.4% for score ≥ 10, p < 0.0001), higher positive predictive value (89.4% versus 78.7% for score ≥ 10) and higher cut-off point accuracy would result.

At 1-year-follow-up, if an EULAR/ACR total score ≥ 13 is adopted, a similar sensitivity (91.0% versus 95.1% for score ≥ 10, p = 0.063), but again a higher specificity (83.2% versus 58.4% for score ≥ 10, p < 0.0001), higher positive predictive value (88.1% versus 75.8% for score ≥ 10) and higher cut-off point accuracy would result.

At first visit, cSLE cases achieved a mean EULAR/ACR total score of 19.64 ± 0.78 and controls 7.93 ± 0.54 (p = 0.002). At 1-year-follow-up, cases had a mean total score of 24.42 ± 0.74 and controls 9.19 ± 0.57 (p = 0.038).

Twenty-nine controls were misclassified as cSLE at first visit if total score ≥ 10 (3 SoJIA, 10 JDM, 2 JSS, 8 MCTD, 1 primary APS, 4 SS and 1 primary vasculitis), against eleven misclassified controls, if total score ≥ 13 (2 SoJIA, 1 JDM, 1 JSS, 6 MCTD, and 1 SS). Eight additional controls were misclassified at 1-year-follow-up if score ≥ 10 (4 JDM, 1 MCTD, 2 SS and 1 primary vasculitis), against 4 other controls if score ≥ 13 (2 JDM and 2 MCTD).

Table 3 displays the global accuracy for each criteria set.

Table 3
Global accuracy for ACR 1997, SLICC and the new EULAR/ACR criteria, at baseline/first visit and 1-year-follow-up

Classification System	Follow-up time	Global accuracy (IC 95%)
ACR 1997	Baseline/First visit	0.830 (0.776-0.884)
	1-year-follow-up	0.933 (0.896-0.969)
EULAR/ACR	Baseline/First visit	0.874 (0.827-0.921)
	1-year-follow-up	0.929 (0.894-0.964)
SLICC	Baseline/First visit	0.910 (0.870-0.949)
	1-year-follow-up	0.952 (0.923-0.982)

• Global accuracy was measured through the area under the ROC curve IC 95-95% confidence interval, ACR American College of Rheumatology, SLICC Systemic Lupus International Collaborating Clinics criteria, EULAR/ACR European League Against Rheumatism (EULAR)/ACR

First visit - ACR 1997 versus SLICC criteria

SLICC criteria exhibited higher sensitivity (89.3% versus 70.5%, p < 0.001), but similar specificity (80.9% versus 83.2% ACR 1997, p = 0.791). SLICC criteria resulted in less misclassifications (30 versus 51, p < 0.001).

First visit - ACR 1997 versus EULAR/ACR total score ≥ 10

EULAR/ACR criteria total score ≥ 10 exhibited higher sensitivity than ACR 1997 (87.7% versus 70.5%, p < 0.001). However, ACR 1997 specificity was higher at first visit (83.2% versus 67.4%, p = 0.004). The number of misclassified patients was similar (44 for EULAR/ACR versus 51 for ACR 1997, p = 0.15).

First visit - ACR 1997 versus EULAR/ACR total score ≥ 13

An EULAR/ACR total score ≥ 13, resulted in no statistically significant difference in sensitivity (76.2% versus 70.5% for ACR 1997, p = 0.189); and specificity (87.6% versus 83.2% for ACR 1997, p = 0.424. EULAR/ACR ≥13 resulted in less misclassifications (40 versus 51, p = 0.023).

First visit - SLICC versus EULAR/ACR total score ≥ 10

Sensitivities were similar at first visit (89.3% for SLICC versus 89.3%, p = 0.687). However, SLICC specificity was higher at first visit (80.9% versus 67.4%, p = 0.008), with less misclassifications (30 versus 44, p = 0.003).

First visit - SLICC versus EULAR/ACR total score ≥ 13

The higher SLICC sensitivity persisted (89.3% versus 76.2%, p < 0.001). There were no differences in specificity (87.6% versus 80.9% for SLICC, p = 0.109). There were less misclassifications for SLICC (30 versus 40, p = 0.032).

1-year-follow-up - ACR 1997 versus SLICC criteria

Sensitivity (97.5% versus 95.1%, p = 0.250), and specificity (76.4% both, p = 0,999) were similar, as well as the number of misclassifications (24 versus 27, p = 0.48).

Global accuracy and cut-off point accuracy were higher for SLICC criteria at both observation periods.

1-year-follow-up - ACR 1997 versus EULAR/ACR total score ≥ 10

EULAR/ACR criteria total score ≥ 10 had similar sensitivity (95.1% for both, p = 0.999). However, ACR 1997 specificity again was higher (76.4% versus 58.4%, p = 0.002), with less misclassifications (43 versus 27, p = 0.006).

1-year-follow-up - ACR 1997 versus EULAR/ACR total score ≥ 13

An EULAR/ACR total score ≥ 13, displayed no statistical significant difference in sensitivity (91.0% versus 95.1% for ACR 1997, p = 0.063), or specificity (83.2% versus 76.4% for ACR 1997, p = 0.146), or in the number of misdiagnosis (26 versus 27, p = 0.816).

1-year-follow-up - SLICC versus EULAR/ACR total score ≥ 10

Sensitivities were similar (97.5% for SLICC versus 95.1%, p = 0.250). However, SLICC specificity was higher (76.4% versus 58.4%, p = 0.001). There were less misclassifications for SLICC (24 versus 43, p < 0.001).

1-year-follow-up-SLICC versus EULAR/ACR total score ≥ 13

The higher SLICC sensitivity persisted (97.5% versus 91.0%, p = 0.008). There were no differences in specificity (83.2% versus 76.4% for SLICC, p = 0.146) and misclassifications (24 versus 26, p = 0.636).

Table 4 (first visit), Table 5 (1-year follow-up) and ^{Additional file 1: Tables S1-S5} summarizes comparative performance measures of the three classification criteria sets.

Table 4
Performance measures for ACR 1997, SLICC and new EULAR/ACR criteria, at baseline/first visit

Classification System	Cut-off point	Sensitivity (IC 95%)	Specificity (IC 95%)	PPV (%)	NPV (%)	Cut-off point accuracy (IC 95%)
ACR 1997	≥ 4	70.5%	83.2%	85.2%	67.3%	75.8
	criteria	(61.6-78.4)	(73.7-90.3)			(69.5-81.4)
EULAR/ACR	Total	87.7	67.4	78.7	80.0	79.2
	score ≥ 10	(80.5-93.0)	(56.7-77.0)			(73.0-84.4)
	Total	76.2	87.6	89.4	72.9	81.0
	score ≥ 13	(67.9-83.5)	(79.0-93.7)			(75.1-86.1)
SLICC	≥4	89.3	80.9	86.5	84.7	85.8
	criteria	(82.5-94.2)	(71.2-88.5)			(80.3-90.2)

• IC 95-95% confidence interval, PPV positive predictive value, NPV negative predictive value, ACR American College of Rheumatology, SLICC Systemic Lupus International Collaborating Clinics criteria, EULAR/ACR European League Against Rheumatism (EULAR)/ACR

Table 5
Performance measures for ACR 1997, SLICC and new EULAR/ACR criteria, at 1-year-follow-up

Classification System	Cut-off point	Sensitivity (IC 95%)	Specificity (IC 95%)	PPV (%)	NPV (%)	Cut-off point accuracy (IC 95%)
ACR 1997	≥ 4 criteria	95.1	76.4	84.7	91.9	87.2
		(89.6-98.2)	(66.2-84.8)			(81.9-91.4)
EULAR/ACR	Total score ≥ 10	95.1	58.4	75.8	89.7	79.6
		(89.6-98.2)	(47.5-68.8)			(73.6-84.8)
	Total score ≥ 13	91.0	83.2	88.1	87.1	87.7
		(84.4-95.4)	(73.7-90.3)			(82.5-91.8)
SLICC	≥ 4 criteria	97.5	76.4	85.0	95.8	88.6
		(93.0-99.5)	(66.2-84.8)			(83.6-92.6)

• IC 95-95% confidence interval, PPV positive predictive value, NPV negative predictive value, ACR American College of Rheumatology, SLICC Systemic Lupus International Collaborating Clinics criteria, EULAR/ACR European League Against Rheumatism (EULAR)/ACR

Discussion

Recently it was published that the new EULAR/ACR could be more sensitive and specific for adult SLE classification (especially for initial classification) than previous criteria sets. Concerning the need for more specific criteria for cSLE classification, we compared three available classification criteria sets. To the best of our knowledge, this is the first study regarding the assessment of performance among three classification systems (ACR 1997, SLICC and especially the new EULAR/ACR criteria) applied exclusively to cSLE patients, in two different observation periods (first visit and 1-year-follow-up).

Few studies have evaluated the performance of different SLE classification criteria in cSLE, showing variables results. For cSLE, three studies demonstrated that SLICC criteria classify patients earlier than ACR 1997 [¹⁸-²⁰]. In the multicentre European study by Sag and colleagues, the sensitivity of SLICC was higher (98.7% versus 85.3%, p < 0.001) but specificity was lower (76.6% versus 93.4%, p < 0.001) compared to ACR 1997, at time of diagnosis [¹⁸]. The lower specificity of SLICC criteria was mainly attributed to the fulfillment by controls with hemolytic-uremic syndrome or JDM [¹⁸]. Our group has previously assessed ACR 1997 versus SLICC classification at first visit and 1-year-follow-up, in a Brazilian single center cSLE cohort. Sensitivity for SLICC was higher than ACR 1997, 82.7% versus 58.0% at first visit (p < 0.001), but similar at 1-year (96.3% versus 91.3%, p = 0.125). Specificity was not significantly different [¹⁹]. The UK juvenile SLE cohort evaluated only sensitivity, and SLICC also were more sensitive than ACR 1997, both at diagnosis (92.9% versus 84.1%, p < 0.001) and at last visit (100% versus 92%, p < 0.001) [²⁰].

Hartman and colleagues [¹⁷] conducted a systematic literature review for studies comparing ACR 1997 and SLICC criteria with clinical diagnosis in adult SLE and cSLE patients with disease duration up to 5 years. A meta-analysis estimated the sensitivity and specificity of these criteria sets and their variables. Four cSLE studies (568 cSLE patients, 339 controls), including our group previous study [¹⁹], were included, showing a higher sensitivity for early classification with SLICC criteria (99.9% versus 84.3%), but lower specificity than ACR 1997 (82.0% versus 94.1%).

In this present study, the lower SLICC specificity was not found, in contrast with most previous studies. Some reasons to explain why SLICC specificity was comparable to ACR1997 and higher than EULAR/ACR ≥ 10 in our study might be a different control group composition, the assessment of performance in distinct observation periods and especially the cut-off point for total EULAR/ACR score derived from our data.

SLICC criteria also exhibited the highest sensitivity for earlier classification (at first visit), in comparison to ACR 1997 (p < 0.001) and EULAR/ACR ≥ 13 (p < 0.001), but similar to EULAR/ACR proposed total score ≥ 10 (89.3% versus 87.7%, p = 0.687).

EULAR/ACR total score cut-off point influenced its performance measures. The selection of a EULAR/ACR total score ≥ 13, as determined by ROC curve analysis, against the initially proposed ≥10 score, resulted in higher specificity, positive predictive value, and accuracy. SLICC criteria exhibited higher global accuracy at both observation periods. Concerning the EULAR/ACR total score cut-off point being compared (whether ≥10 or ≥ 13), application of SLICC criteria still better scored in cut-off point accuracy both at first visit and at 1-year-follow-up, in our cSLE population.

The expanded scope of clinical (especially cutaneous and CNS manifestations) and immunologic manifestations included in SLICC criteria, besides the allocation of cytopenias and antibodies into separate criterion might have allowed a higher sensitivity, especially earlier in the disease course. The SLICC rule to classify only patients that have at least one immunologic criterion, preventing SLE classification based solely on clinical manifestations, and the ending of the “double counting” of photosensitive malar rash as two criteria (as in ACR 1997), may have increased SLICC specificity.

The differences in performance among those three sets of classification criteria might be due to changes either in the definition of organ involvement, to cut-off points for positive autoantibodies or to the form in which several clinical manifestations and laboratory parameters are gathered within each criteria set.

We are aware that this study is limited by the retrospective design and the relatively small number of patients and controls included. However, the extraction and collection of data by two authors, and the confirmation by the other three authors, before criteria set scoring, minimized this methodological limitation.

Second, a structural problem in designing and validating classification criteria, and thereby in interpreting our study, is an inherent lack of an objective diagnosis as the standard of reference other than clinical diagnosis, so that the treating physician's diagnosis, is still adopted by most studies. We also decided to use as our standard of reference, the diagnosis consolidated during continuous follow-up of all patients by a group of highly experienced pediatric rheumatologists. It may be argued that this could lead to inconsistency; yet, it does allow evaluation of classification criteria in a real-world setting. Finally, we decided to compose our control group with rheumatic diseases that impose difficult differential diagnosis with JSLE, simulating the daily clinical practice; however, we did not include controls with undifferentiated disease or overlapping syndromes.

Considering the peculiarities of cSLE (more severe presentation and course, higher rates of disease activity and cumulative damage, higher frequency of atypical and constitutional manifestations) and taking into account that these three sets of criteria were developed in adult SLE, it should be considered modifications to increase early sensitivity and specificity for cSLE classification.

Conclusion

In this cSLE population, SLICC criteria better scored at first visit and 1-year-follow-up. The adoption of a EULAR/ACR total score ≥ 13 in this study, against the initially proposed ≥10 score, was most appropriate to classify cSLE. Further studies are necessary to address if SLICC criteria might allow fulfillment of cSLE classification earlier in disease course and may be more inclusive of cSLE subjects for clinical studies.

• Funding
  There are no sources of funding for the research to be declared.
• Availability of data and materials
  The datasets used and analyzed during the current study are available from the corresponding author on reasonable request.
• Ethics approval and consent to participate
  The local Research Ethics Committee approved the study protocol before study commencement, under Number 2.421.080, on December 7th, 2017.
• Consent for publication
  Not applicable.
• Publisher's Note
  Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Abbreviations

• APS  Antiphospholipid syndrome
• JDM  Juvenile dermatomyositis
• JSS  Juvenile systemic scleroderma
• MCTD  Mixed connective tissue disease
• SoJIA  Systemic onset juvenile idiopathic arthritis
• SS  Sjogren syndrome

Acknowledgements

We acknowledge Dr. Mariana Aires M.D. PhD and Dr. Elaine Sobral M.D. PhD, for helpful inputs.

Additional file

Additional file 1:Table S1. Comparison among ACR 1997 and SLICC criteria. Table S2. Comparison among ACR 1997 and EULAR/ACR criteria (total score ≥ 10). Table S3. Comparison among ACR 1997 and EULAR/ ACR criteria (total score ≥ 13). Table S4. Comparison among EULAR/ACR (total score >10) and SLICC criteria. Table S5. Comparison among EULAR/ACR (total score >13) and SLICC criteria. (DOC 69 kb)

References

• ¹
  Hedrich CM, Smith EMD, Beresford MW. Juvenile-onset systemic lupus erythematosus (jSLE)-Pathophysiological concepts and treatment options. Best Pract Res Clin Rheumatol. 2017;31:488-504.
• ²
  Harry O, Yasin S, Brunner H. Childhood-Onset Systemic Lupus Erythematosus: A Review and Update. J Pediatr. 2018;196:22-30 e2.
• ³
  Brunner HI, Gladman DD, Ibanez D, Urowitz MD, Silverman ED. Difference in disease features between childhood-onset and adult-onset systemic lupus erythematosus. Arthritis Rheum. 2008;58:556-62.
• ⁴
  Bundhun PK, Kumari A, Huang F. Differences in clinical features observed between childhood-onset versus adult-onset systemic lupus erythematosus: A systematic review and meta-analysis. Medicine (Baltimore). 2017;96:e8086.
• ⁵
  Hochberg MC. Updating the American College of Rheumatology revised criteria for the classification of systemic lupus erythematosus. Arthritis Rheum. 1997;40:1725.
• ⁶
  Larosa M, Iaccarino L, Gatoo M, Punzi L, Doria A. Advances in the diagnosis and classification of systemic lupus erythematosus. Expert Rev Clin Immunol. 2016;12:1309-20.
• ⁷
  Petri M, Orbai AM, Alarcon GS, Gordon C, Merrill JT, Fortin PR, et al. Derivation and validation of the systemic lupus international collaborating clinics classification criteria for systemic lupus erythematosus. Arthritis Rheum. 2012;64:2677-86.
• ⁸
  Aringer M, Dorner T, Leuchten N, Johnson SR. Toward new criteria for systemic lupus erythematosus—a standpoint. Lupus. 2016;25:805-11.
• ⁹
  Tedeschi SK, Johnson SR, Boumpas DA, Daikh D, Dorner T, Jayne D, et al. Developing and refining new candidate criteria for SLE classification: an international collaboration. Arthritis Care Res (Hoboken). 2018;70:571-81.
• ¹⁰
  Aringer M, Costenbader KH, Brinks R, Boumpas D, Daikh D, Jayne D, et al. Validation of New Systemic Lupus Erythematosus Classification Criteria. In: 2018 ACR/AHRP annual meeting. Chicago;2018. https://acrabstracts.org Accessed 24 Oct 2018.
  » https://acrabstracts.org
• ¹¹
  Aringer M, Dörner T. Systemic Lupus Erythematosus (SLE) - New Classification Criteria [in German]. Dtsch Med Wochenschr. 2018;143:811-4.
• ¹²
  Mosca M, Costenbader KH, Johnson SR, Lorenzoni V, Sebastiani GD, Hoyer BF, et al. How do patients with newly diagnosed systemic lupus erythematosus present? A multicenter cohort of early systemic lupus erythematosus to inform the development of new classification criteria. Arthritis Rheumatol. 2019;71(1):91-8.
• ¹³
  Amezcua-Guerra LM, Higuera-Ortiz V, Arteaga-Garcia U, Gallegos-Nava S, Hubbe-Tena C. Performance of the 2012 SLICC and the 1997 ACR classification criteria for systemic lupus erythematosus in a real-life scenario. Arthritis Care Res (Hoboken). 2015;67:437-41.
• ¹⁴
  Inês L, Silva C, Galindo M, López-Longo FJ, Terroso G, Romão VC, et al. Classification of Systemic Lupus Erythematosus: Systemic Lupus International Collaborating Clinics versus American College of Rheumatology Criteria. A Comparative Study of 2.055 patients from a real- life, international systemic lupus erythematosus cohort. Arthritis Care Res (Hoboken). 2015;67:1180-5.
• ¹⁵
  Ighe A, Dahlström O, Skogh T, Sjöwall C. Application of the 2012 Systemic Lupus International Collaborating Clinics classification criteria to patients in a regional Swedish systemic lupus erythematosus register. Arthritis Res Ther. 2015;17:3.
• ¹⁶
  Oku K, Atsumi T, Akiyama Y, Amano H, Azuma N, Bohgaki T, et al. Evaluation of the alternative classification criteria of systemic lupus erythematosus established by systemic lupus international collaborating clinics (SLICC). Mod Rheumatol. 2018;28:642-8.
• ¹⁷
  Hartman EAR, Van Royen-Kerkhof A, Jacobs JWG, Welsing PMJ, Fritsch-Stork RDE. Performance of the 2012 systemic lupus international collaborating clinics classification criteria versus the 1997 American College of Rheumatology Classification Criteria in adult and juvenile systemic lupus erythematosus. A systematic review and meta-analysis. Autoimmun Rev. 2018;17:316-22.
• ¹⁸
  Sag E, Tartaglione A, Batu ED, Ravelli A, Khalil SM, Marks SD, Ozen S, et al. Performance of the new SLICC classification criteria in childhood systemic lupus erythematosus: a multicentre study. Clin Exp Rheumatol. 2014;32:440-4.
• ¹⁹
  Fonseca AR, Gaspar-Elsas MIC, Land MGP, de Oliveira SKF. Comparison between three systems of classification criteria in juvenile systemic lupus erythematous. Rheumatology (Oxford). 2015;54:241-7.
• ²⁰
  Lythgoe H, Morgan T, Heaf E, Lloyd O, Al-Abadi E, Armon K, et al. Evaluation of the ACR and SLICC classification criteria in juvenile-onset systemic lupus erythematosus: a longitudinal analysis. Lupus. 2017;26:1285-90.
• ²¹
  Schmajuk G, Hoyer BF, Aringer M, Johnson SR, Daikh DI, Dörner T. Multicenter Delphi Exercise Reveals Important Key Items for Classifying Systemic Lupus Erythematosus. Arthritis Care Res (Hoboken). 2018;70:1488-94.
• ²²
  Leuchten N, Hoyer A, Brinks R, Schoels M, Schneider M, Smolen J, et al. Performance of anti-nuclear antibodies for classifying systemic lupus erythematosus: a systematic literature review and meta-regression of diagnostic data. Arthritis Care Res (Hoboken). 2018;70:428-38.
• ²³
  Pisetsky DS, Lipsky PE. The Role of ANA Determinations in Classification Criteria for SLE. Arthritis Care Res (Hoboken). 2018. https://doi.org/10.1002/acr.23559
  » https://doi.org/10.1002/acr.23559
• ²⁴
  Petty RE, Southwood TR, Manners P, Baum J, Glass DN, Goldenberg J, et al. International league of associations for rheumatology classification of juvenile idiopathic arthritis: second revision, Edmonton, 2001. J Rheumatol. 2004;31:390-2.
• ²⁵
  Bohan A, Peter JB. Polymyositis and dermatomyositis (first of two parts). N Engl J Med. 1975;292:344-7.
• ²⁶
  Zulian F, Woo P, Athreya BH, Laxer RM, Medsger TA Jr, Lehman TJ, et al. The Pediatric Rheumatology European Society/American College of Rheumatology/European League against Rheumatism provisional classification criteria for juvenile systemic sclerosis. Arthritis Rheum. 2007;57: 203-12.
• ²⁷
  Bartunkova J, Sediva A, Vencovsky J, Tesar V. Primary Sjogren's syndrome in children and adolescents: proposal for diagnostic criteria. Clin Exp Rheumatol. 1999;17:381-6.
• ²⁸
  Miyakis S, Lockshin MD, Atsumi T, Branch DW, Brey RL, Cervera R, et al. International consensus statement on an update of the classification criteria for definite antiphospholipid syndrome. J Thromb Haemost. 2006;4:295-306.
• ²⁹
  Kasukawa R, Tojo T, Miyawaki S. Preliminary diagnostic criteria for classification of mixed connective tissue disease. In: Kasukawa R, Sharp GC, editors. Mixed connective tissue disease and antinuclear antibodies. Amsterdam: Elsevier; 1987. p. 41-7.
• ³⁰
  Šimundić AM. Measures of Diagnostic Accuracy: Basic Definitions. EJIFCC. 2009;20(19):203-11.

Publication Dates

• Publication in this collection
  29 July 2019
• Date of issue
  2019

History

• Received
  07 Nov 2018
• Accepted
  01 May 2019
• Published
  15 May 2019