A systematic review and meta-analysis of the diagnostic accuracy of the Phototest for cognitive impairment and dementia

The recently developed Phototest is a simple, easy and very brief test for detecting cognitive impairment or dementia. Objective To evaluate the diagnostic accuracy of the Phototest for detecting cognitive impairment or dementia. Methods We used a manually created database to search for studies evaluating the Phototest diagnostic yield and performed an initial meta-analysis to determine sensitivity (Sn) and specificity (Sp) of diagnostic parameters. We also performed a second meta-analysis of individual participant data. Results In total, 6 studies were included in the meta-analysis. For dementia, Sn was 0.85 (95% CI, 0.82-0.88) and Sp 0.87 (95% CI, 0.85-0.99); for cognitive impairment, Sn was 0.80 (95% CI, 0.77-0.92) and Sp 0.88 (95% CI, 0.86-0.90). In the individual data meta-analysis, 1565 subjects were included, where best cut-off points for dementia and for cognitive impairment were 26/27 (Sn=0.89 (95% CI 0.85-0.91), Sp=0.84 (95% CI, 0.82-0.91)) and 28/29 (Sn=0.79 (95% CI, 0.76-0.81), Sp=0.88 (95% CI, 0.86-0.90)), respectively. Conclusion Phototest has good diagnostic accuracy for dementia and cognitive impairment. It is brief, simple and can be used in illiterate persons. This makes it suitable for use in primary care settings and/or in subjects with low educational level.


INTRODUCTION
T he Phototest (http://www.phototest.es) is a recently developed cognitive test with theoretical advantages over other available dementia screening tests: it is simple and very brief (<3 minutes), can be applied to illiterates, and has results that are not influenced by the subject's educational level. 1 The test comprises three parts. 2 First, a naming task with a sheet including the six col-or photographs of common objects in prototypic position corresponding to different categories (games, vehicles, musical instruments, clothes and eating utensils) is administered. These pictured objects vary in frequency, from high (car) to low frequency (trumpet); in semantic set size, from broad (fruit) to narrow (cutlery); as well as in whether they are prototypical (spoon) or atypical (shoes) elements in a given semantic field. The sheet is then removed. The second task is a verbal fluency test in which participants are asked to say as many oppositegender names as possible in thirty seconds, and then same-gender ones during the same time period. Unlike other frequently used verbal fluency tests (e.g. animal verbal fluency), this task was shown to be uninfluenced by educational level. 3 Following the verbal fluency test (names of people), which also has a significant distractor effect between naming and recall tasks; subjects are asked to freely recall the six photographs in any order. After 20 seconds, the category cues are presented to elicit cued recall of only those items that are not retrieved by free recall. In summary, the Phototest assesses multiple cognitive domains including language (naming objects), executive functions (verbal fluency) as well as episodic memory (free recall and cued recall), which show high sensitivity for detecting cognitive impairment (CI) in general, and Alzheimer disease (AD) in particular. 4 Several studies have assessed the diagnostic accuracy of the Phototest for detecting cognitive impairment and dementia in different settings and sites; however, to date, no meta-analysis of these studies has been performed. The objective of this systematic review and meta-analysis was to evaluate the diagnostic accuracy of the Phototest for detecting cognitive impairment and/or dementia.

METHODS
Search criteria for systematic review. A database of published and unpublished literature was assembled from systematic searches of electronic sources, hand searching, and consultation with experts in the field. The following databases were searched: MEDLINE, PsycINFO, EmBASE, SCIELO, and IBECS. In addition, information on studies in progress, unpublished research or research reported in the grey literature was sought by searching a range of relevant databases including Inside Conferences and Dissertation Abstracts. All studies published from 1 January 2004 through 31 December 2013 that evaluated the diagnostic accuracy of the Phototest for detecting cognitive impairment or dementia, were identified. The search was restricted to English and Spanish language literature. With use of a Boolean strategy, cross searching of the following four categories was done: 1-test ("Fototest" OR "Test de las Fotos" OR "Phototest" OR "Photo-Test"), 2-disease ("cognitive impairment" OR "dementia" or "Alzheimer´s disease"), 3-estimates of diagnostic test accuracy ("sensitivity" OR "specificity" OR "accuracy"), and search terms representing screening tests ("screening" OR "detection"). Furthermore, additional articles were identified by manually searching the authors' own literature databases.
Inclusion criteria and selection process. To be included in the review, studies had to meet the following criteria: [1] population: cognitively healthy older adults or adults with cognitive impairment or dementia according to validated reference standard diagnostic criteria; [2] outcome: diagnostic accuracy of the Phototest; and [3] reported data: sensitivity (Sn) and specificity (Sp) values; when unavailable, raw data from the articles were used to construct 2x2 tables. Authors of individual reports were contacted to verify data extracted from the original database and to provide supplementary information pertaining to the criteria used for diagnosing cognitive impairment or dementia.
We developed a data extraction sheet. One review author extracted, or calculated from each study, data on the sensitivity and specificity of the Phototest while a second author checked the extracted data.

Meta-analysis.
For included studies, our primary outcomes of interest were Sn and Sp at a given cut-off for the Phototest. We did not focus on positive and negative predictive values because the prevalence of cognitive impairment varied widely across studies. We synthesized results for test performance to detect: [1] dementia; and [2] cognitive impairment (MCI and/or dementia). We ran a random effect meta-analysis (Dersimonian-Laird method) for sensitivity and specificity for both groups, using the Meta-Disc11 5 program. Heterogeneity was assessed by the Cochran Q test. We also performed a meta-analysis of participants including data from authors' own databases, for which we estimated the area under the ROC curve (aROC), Youden index, Sn and Sp values, and positive and negative likelihood ratios (+LR, -LR), and 95% confidence intervals were calculated for all studied variables. Finally, specific LRs were calculated for different score intervals.
The PRISMA-statement was followed for reporting items of this systematic review and meta-analyses. 6

RESULTS
Literature search. The literature search yielded a total of 10 potentially relevant articles. We also included 2 addi-Carnero-Pardo C, et al.
Phototest for cognitive impairment and dementia tional studies conducted in Argentina (an unpublished relevant study 7 and another ongoing study) 8 ( Figure 1). In total, the articles identified were: two letters to the editor, 9, 10 one normative study 11 and nine diagnostic test accuracy studies. 1,2,7,8,[12][13][14][15][16] Eight references met the inclusion criteria for this systematic review. Four studies were excluded for the following reasons: letter to editor (n=2), 9,10 duplicated study (n=1), 14 and normative data study (n=1). 11 Finally, two studies were excluded from the meta-analysis since they involved a preliminary version of the Phototest (n=2). 1,7 Diagnostic criteria. All studies from Spain met the Diagnostic and Statistical Manual of Mental Disorders-Fourth Edition, Text Revision (DSM-IV-TR) 17 criteria for dementia diagnosis, and the recommendations from the Spanish Neurological Society for mild cognitive impairment diagnosis (MCI) 18 . In the Argentine study, only subjects who met the National Institute of Neurological and Communicative Disorders and Stroke-Alzheimer's Disease and Related Disorders Association (NINDS-ADRDA) 19 criteria for probable Alzheimer-type dementia, the DSM-IV-TR 17 criteria for dementia, and the conventional Petersen criteria 20 for single domain amnestic MCI were included.
Characteristics of included studies. The systematic review included 8 articles (n=1939 participants, ranging from 60 to 589 subjects). The details of the included studies are summarized in Table 1 and explained below: Preliminary Study 1 -Case-control study providing "proof of concept" of the Phototest. This early version of the test used a sheet with 6 photographs of objects, different to those in the current version, therefore the study was excluded from the current analysis. AD8 Study 16 -Cross-sectional, prospective study conducted at a Cognitive-Behavioral Neurology Unit, which served as validation of the Spanish version of the AD8 questionnaire (20). In this study, the Phototest was used as a short cognitive reference test to compare and assess construct validity of the AD8 questionnaire. 21 Argentine-2 Study 8 -Cross-sectional study of elderly clinical patients attending two memory clinics in Argentina, selected by convenience sampling of consecutive patients suspected of cognitive impairment or dementia.
Cut-off points. The optimal cut-off scores for identifying dementia in the studies included varied between 24/25 and 26/27 (positive/ negative). For discriminating between control and cognitive impairment subjects, the optimal cut-off score of the Phototest was 28/29 in the Spanish study and 30/31 in the Argentine sample. Apart from published information, all individual data were available for 5 out of the 6 included stud-

Meta-analysis
Dementia -The overall prevalence of dementia for studies in different settings was 30.5 %. Sn and Sp diagnostic parameters for dementia are shown in Table 1.  Table 2.
The meta-analysis of individual data also allowed estimation of specific LRs for different cut-off points or intervals for dementia (Table 4) and cognitive impairment (Table 5).

DISCUSSION
In this systematic review, we found a total of 8 studies concerning Phototest diagnostic accuracy, 6 of which were included in the meta-analysis, 2,8,12,13,15,16 whereas 2 studies were excluded because they employed a preliminary pilot version of the test. 1 In the latter analysis, we calculated clinically relevant score intervals and their corresponding LRs. This allowed estimation of pooled post-test probability for known sample prevalence. 22 Studies were heterogeneous regarding number of subjects included, variable settings, cut-off points used and quality of data. They were however, highly homogeneous with respect to diagnostic criteria since, except for the Argentine-2 study, authors used the same diagnostic criteria, namely: DSM-IV-TR 17 for dementia and MCI criteria from the Spanish Neurological Society 18 for cognitive impairment. The Argentine-2 study only included patients with probable Alzheimer´s disease and single domain amnestic MCI.
It is known that the Mini-Mental State Examination (MMSE) 23 is the most commonly used cognitive screening test. However, a meta-analysis of the accuracy of the MMSE 24 revealed its very limited value in detecting dementia (Sn=0.77, 95% CI 0.70-0.83; Sp=0.90, 95% CI 0.82-0.95), and particularly MCI (Sn=0.67, 95% CI 0.50-0.82; Sp=0.78, 95% CI 0.62-0.90). These results are in line with a side-by-side comparison of the two instruments, in which the Phototest proved to be more efficient due to superior effectiveness and lower cost than the MMSE. 13 In addition to lower effectiveness, the MMSE has numerous other limitations (lack of standardization, application time, influence of socioeducational variables and copyright). These drawbacks have led one of the authors to suggest this could be the right time to reject this test. 25 The Eurotest 26 is another instrument which can be used in illiterate persons that is independent of socioeducational factors and has shown similar diagnostic accuracy to the Phototest in a recent meta-analyses 27  ). Another study with a side-by-side comparison of the Eurotest and Phototest showed the same effectiveness. 14 If we consider that the Eurotest requires double the time to apply (6 to 8 min) than the Phototest (<3 min), despite being equally effective, the latter proves to be more efficient and useful in clinical practice.
The Clock Drawing Test 28 is another brief and widely-used cognitive test. However, in a recent systematic review, 29 a quantitative meta-analysis was unfeasible due to the large variety of correction systems applied in publications. Two recent studies have shown modest utility both in specialized (aROC=0.88 for dementia, aROC=0.82 for cognitive impairment) 30 and in primary care settings (aROC=0.84 for CI). 31 These poor results may be linked to low educational level of the study populations, since it is known that education greatly influences test results. 29,32 The main limitations of this meta-analysis are the study heterogeneity and the small sample size of some of the studies included. Its strengths include the homogeneity of the diagnostic criteria used, the varied study locations which comprised multiple clinical settings, and above all, the individual participant data analysis that included data from all studies, bar one. This metaanalysis allowed robust estimation of diagnostic accuracy parameters for either point estimate or interval results.
In conclusion, this meta-analysis showed that the Phototest offers adequate diagnostic accuracy for cognitive impairment, and particularly for dementia, that is similar or superior to other instruments widely used in our milieu. Additionally, it is simple, brief, uninfluenced by educational variables and can even be used in individuals who are illiterate. These advantages make it attractive for use in populations with low educational level and/or in time-limited settings such as primary care.
Financial disclosure. C. Carnero-Pardo is the creator of the Phototest and has received speaker and consultancy fees from companies involved in the manufacturing and marketing of drugs for dementia -Janssen Cilag, Pfizer, Eisai, Esteve, Novartis, Lundbeck and Andrómaco. The other authors have no relevant financial interests in the manuscript. The authors did not receive remuneration for the production of this manuscript.