Addenbrooke’s Cognitive Examination-III (ACE-III) test as a new potential tool for screening of cognitive dysfunctions in the course of multiple sclerosis

INTRODUCTION

Cognitive dysfunction affects 40-70% of patients suffering from multiple sclerosis (MS) [1]. The assessment of cognitive functioning level in a patient allows to monitor the course of the disease, to plan appropriate therapeutic interventions and to describe the cognitive profile for a given disease for research purposes [1]. Patients with MS qualified for the treatment program regularly undergo an assessment of daily functioning, including cognitive functioning, which is why there is a need to analyse the available tools to assess this aspect. The purpose of the study was to analyse the usefulness of the Addenbrooke’s Cognitive Examination-III (ACE-III) test in the diagnosis of cognitive dysfunctions in the course of MS. The test selection criteria included mainly the scope of the studied cognitive functions, the time of the examination, the simplicity of performance, and the availability of the test. Due to the lack of the Polish language version, it was impossible to use the PASAT (Paced Auditory Serial Attention Test) commonly used to study cognitive functions in patients with MS in the English-language projects.
The ACE-III was created for primary neurodegenerative diseases and patients over 50 years of age. The test includes three parallel versions, allowing the examiner to repeat the tests in order to monitor the cognitive functioning as well as the progression of the disease. The test is available free of charge and without restrictions, also for doctors and other broadly defined health care professionals. Learning the technique is simple and takes few hours, free training materials are available. The ACE-III has more extended assessment scale (from 0 to 100 points) than other standard screening tests, e.g. MoCA (Montreal Cognitive Assessment) or MMSE (Mini Mental State Examination) (both from 0 to 30 points). An abbreviated version of the ACE-III (M-ACE) is available, also in three parallel versions, which is a good alternative to the longer version of the test. The scope of the studied cognitive functions and their influence on the summary results in ACE and M-ACE are presented in Table 1. Due to the sensitivity and specificity in the detection of mild cognitive dysfunctions and a special profile assessing primarily functioning of the patients, the ACE-III seems to be a potentially useful tool in the assessment of MS. Although version III of the ACE test is the latest modification, it has been verified by five standard tests: WAIS-DS (Wechsler Adult Intelligence Scale-Digit Scale), RAVLT (Rey Auditory Verbal Learning), SydBAT (Sydney Language Battery), ROCFT (Rey-Osterrieth Complex Figure Test), and FTDFRS (Frontotemporal Dementia Functional Rating Scale), which confirmed the statistical correlation of ACE-III results with the results of the aforementioned tests [2]. Satisfactory correlations of the results of the ACE-III subscales with the results of neuropsychological tests assessing particular areas of cognitive functioning were also confirmed in another independent study [3]. According to the literature reports, the test can be considered as credible. In the case of neuropsychological assessment of a patient with MS, in addition to the aforementioned PASAT, the most frequently used tests in scientific research are, among others, SDMT (Symbol Digit Modalities Test), TMT (Trail Making Test), verbal fluency tests, WCST (Wisconsin Card Sorting Test), SRT (Selective Reminding Test) and SPART (10/36 Spatial Recall Test) [1]. However, most of these tools have serious limitations in their use in medical practice, i.e. they are available only to a psychologist and often for a fee, or they do not have the Polish language version, and the time of their implementation is long. In terms of all those aspects, the ACE-III may be a useful screening tool for a physician, but it has no Polish standardisation.

PURPOSE

The purpose of the project was to assess the usefulness of the ACE-III as a screening tool in detecting cognitive dysfunctions in MS patients by determining the suggested cut-off points for individual cognitive functions, the result of the ACE-III total score and the M-ACE score. The study was also an attempt to characterise the profile of cognitive dysfunctions presented by patients with MS.

METHODS

Subject and study design

Twenty eight patients with relapsing-remitting MS and 24 controls were enrolled for the study (Table 2). There was no significant difference in age, gender, education level and handedness distribution between the controls and MS patients (Table 2).
The inclusion criteria for MS patients were the diagnosis of relapsing-remitting MS according to the McDonald’s 2010 criteria, successful treatment with immunomodulatory drugs, age 18-65, no relapse or corticosteroid use within 30 days before or after ACE-III examination. The examination of cognitive functions was accompanied by a clinical-radiological evaluation (Expanded Disability Status Scale-EDSS and MRI examination). The aim of the complex assessment was to exclude the ineffectiveness of treatment and the current relapse. The control group included 24 patients with no confirmed neurological disorders. Lesions in the central nervous system of controls were excluded by MRI. Only patients and controls whose native language was Polish were qualified for the study.
Exclusion criteria in both groups were the diagnosis of a psychiatric disorder that may reduce cognitive functioning, especially mood disorders (based on Beck’s Depression Inventory); medicines, including neuroleptics, antiepileptic drugs, antidepressants or mood stabilisers; history of alcohol or psychoactive drugs addiction.

Functional assessment

The assessment of cognitive functions has been carried out using the A version of the ACE-III form. The study has been carried out in a separate room, in optimal lighting conditions, in silence, at the same time of the day, including the supplies necessary for a given patient for optimal vision or hearing correction in everyday conditions. The individual cognitive functions, the results of the ACE-III test (maximum 100 points) and M-ACE (maximum 30 points) have been assessed. During the study, special attention has been paid to the occurrence of motor deficits or sight and hearing deficits that prevented the test from being performed correctly. No patient was excluded from the study because of the above. Additionally, the psychological examination was each time preceded by a neurological examination and a subject-oriented examination aimed at exclusion of conditions potentially affecting the results of psychological assessment (sleepless night, malaise, infection, post-lumbar puncture syndrome, other painful symptoms, Uhthoff phenomenon).
The local bioethical commission issued a consent form for the study and all subjects gave written informed consent.

Statistical analysis

The obtained results were subjected to a standard statistical analysis extended with the ROC curve. ROC curves are used to assess the prognostic ability of the analysed cognitive functions in distinguishing the two studied groups and determine the cut-off points for the method in the case of MS patients. Their use allows to determine the optimal cut-off points for the method for individual cognitive functions, the result of the ACE-III total score and M-ACE score. The ROC curve provides a combined description of the sensitivity and specificity of the classifier. The course of the ROC curve allows to infer the forecast value of the analysed variable based on the AUC value in accordance with the principle:
• AUC < 0.5 – the factor does not differentiate patients (equivalent to random allocation);
• 0.5 ≥ AUC < 0.6 – the factor very poorly differentiates patients;
• 0.6 ≥ AUC < 0.7 – the factor differentiates patients adequately;
• 0.7 ≥ AUC < 0.8 – the factor differentiates patients satisfactorily;
• 0.8 ≥ AUC < 0.9 – the factor differentiates patients very well;
• AUC ≥ 0.9 – the factor differentiates patients perfectly.
The AUC > 0.7 result was found satisfactory in the study. Statistically significant were results for p < 0.05.

RESULTS

All cognitive functions had lower values in the group of MS patients. The results are presented in Table 3.
For all results, ROC curves were determined along with marked optimal cut-off point based on the Youden index value (Figure I).
The highest AUC values were obtained for total score, memory, M-ACE and fluency – all of these values are statistically significant. Each ROC curve was given basic statistics describing the curve (Table 4): area under the ROC curve (AUC), standard error (SE), p significance level and 95% CI limits for the area under the curve.
Using the Youden index based on the drawn ROC curves, optimal cut-off points were determined, for which sensitivity, specificity, positive and negative predictive value (PPV, NPV) and accuracy (ACC) were determined. The results are presented in Table 5.
Variables along with the determined cut-off values can be treated as a diagnostic test allowing to predict the occurrence of the phenomenon, i.e. in the considered case, belonging to a group of patients with MS, we take into account the cut-off point with the ≤ sign (e.g. attention ≤ 17; memory ≤ 24; etc.).

DISCUSSION

Other screening tests than ACE-III, such as MoCA and MMSE, are available in Polish. However, none of those tests offers such extended scoring (rating scale up to 100 points in ACE-III) and the possibility of using the short version of the test (M-ACE); in the case of MoCA test, such shortened version is still in progress. Hsieh et al. [4] emphasise that M-ACE is more sensitive than MMSE and less susceptible to the ceiling effect. The ACE-III test correlates with the standard diagnostic neuropsychological tests used in the assessment of attention, language, verbal memory, and visuospatial function as well as, with the previous version, ACE-Revised (ACE-R), taking into account the level of sensitivity and specificity [5]. According to studies, MoCA and ACE-R are characterised by good sensitivity and diagnostic specificity for mild cognitive dysfunctions, while MMSE is characterised by a ceiling effect [6]. The ACE-R test is also treated by clinicians as an extension of cognitive performance assessment when the results of shorter tests are inconclusive [7]. In addition, the ACE-III test is considered the most sensitive tool for assessing daily functioning of a patient, and especially the subscale for the assessment of language skills as significantly related to the initiation of activity by the patient [8]. The results obtained in the presented study with the ACE-III revealed lower levels of all cognitive functions in the group of patients with MS. Statistically significant differences (p < 0.05) were demonstrated for the results of the ACE-III total score, memory, M-ACE and verbal fluency. With the cut-off level of 97 points in total score, the test was characterised by a satisfactory sensitivity and specificity. In terms of memory and fluency, with cut-off points of 24 and 12 points respectively, it was characterised by sensitivity close to 60% and specificity over 90%. The ACE-III presents high diagnostic accuracy and good sensitivity and diagnostic specificity in the assessment of cognitive dysfunctions. However, the study did not show satisfactory sensitivity for the optimal cut-off point determined by statistical methods in the linguistic subscale for the Polish version of the A form of ACE-III. This discrepancy may result from language differences of the compared versions. Thanks to the creation of three versions of the test, differing in the content of tasks relying on memorising, it allows to avoid the patient’s learning effect and enables repeating the test for control assessment (suggested 6 months apart). A good summary are the comparative studies of MMSE, MoCA and ACE-III, where the ACE-III obtained the highest diagnostic accuracy [9].
The study confirmed the deteriorated functioning of the MS patients in terms of memory, verbal fluency and general cognitive functioning (ACE-III total score, M-ACE). The results obtained in the study with the ACE-III coincide with the literature conclusions on cognitive dysfunctions in MS and are an attempt to characterise the profile of cognitive dysfunction presented by patients with MS. Following the research aiming to find a specific profile of cognitive functioning of patients with MS, it’s important to note the results of the meta-analysis of DeSousa et al. [10]. The authors emphasise the occurrence of attention disorders, abstract thinking disorders, reduced manual speed and reduced manual skills, deficits in memory processes and language abilities in patients with MS. Another meta-analysis by Ferrier [11] emphasises the occurrence of memory disorders, concentration of attention, processing speed as well as executive functions. Additionally, it has been assessed that patients with primary progressive multiple sclerosis have more severe cognitive dysfunctions than patients with relapsing-remitting MS [12]. A lot of research has been conducted to find a specific profile of the cognitive functioning of the patient with depression [13–15]. Cognitive dysfunction attributed to mood disorders may mimic impaired cognition in the course of MS. The group of MS patients recruited for this study was treated in an immunomodulatory treatment program and depression was an exclusion criteria for the treatment (using Beck’s Inventory Scale). In addition, a subject-based study was conducted, the purpose of which was subjective assessment of the patient regarding his current level of functioning. ACE-III usage should be supported with tools to assess mood disorders.
Undoubtedly, the analysis of the cognitive functioning of a patient with MS is the resultant of many factors. Literature emphasises that such important factors may include age, gender and education [16, 17]. According to the literature reports, the total score and diagnostic accuracy of ACE-III depend on the level of education [9, 16]. The length of education is a modifiable factor conditioning the cognitive brain reserve [18]. Considering the correlation between brain atrophy in MS patients with cognitive dysfunctions, one should remember the number of years of education affects this dependence. Considering the influence of education, in this study, the groups did not differ statistically significantly (p < 0.59). In the analysed groups of the MS patients and controls, also the median age did not differ significantly. This is important in the study of populations with large age discrepancies, in which – due to the lower availability of education (e.g. in Europe during the Second World War) – these results may be falsely stratified and suggest greater than the actual impact of age and gender (historically worse access to education for women) on the results of the test. However, the small number of patients may form a limitation of the study.
Other limitations of the study may be the prevalence of women, which is a characteristic of MS. Different efficiency of cognitive functions can, therefore, be an exponent of the percentage of women in the study and not just an exponent of the dysfunctions resulting from the course of MS per se. From this perspective, the predominance of women in groups may translate into the display of higher results in fluency, where women achieve higher scores in relation to the visuospatial functions, where, according to literature, an advantage is noted in men [19]. Literature emphasises the strong relationship between the cognitive functioning and mood disorders, understood as a manifestation of dysfunction of the same neuronal mechanism [20]. In addition, deterioration of the neurologic condition may influence psychological equilibrium and eventually affect the cognitive performance [20]. The study may be limited by the relatively good neurological status of the MS patients group. In the presented study, the neurological status of patients was assessed with the EDSS. During the examination with the ACE-III, particular attention was paid to the potential difficulties with complete execution of the test. None of the patients were disqualified for this reason. Retrospective analysis of EDSS protocols confirmed that in cases of patients with the largest neurological deficits – the highest score in EDSS (6.5 points) – the result corresponded to the deficit related to muscular strength of the lower limbs. It seems that in this aspect, the examined group was not fully representative for patients with MS in general. A number of incomplete tests were to be expected due to the deficit in muscular strength of the dominant upper limb. In the case of a comparative analysis of ACE-III, MoCA and MMS tests in patients after stroke, a significant number of partially completed tests were obtained (14/51, 20/51 and 22/51 respectively) [21]. ACE-III was characterised by the highest completeness of the execution. The addressed problem may be significant in the control assessment of patients with MS. Despite three existing parallel versions of the test and potential utility in the long-term assessment, its use may be limited in patients manifesting from the beginning the paresis of the dominant upper limb or developing such a deficit during the course of the disease. The median duration of the disease in the studied group of patients with MS was 5 years, which is a relatively short time. It is possible that in the group of patients suffering from MS longer, the number of incomplete tests would be significant.
Moreover, the interpretation of statistics obtained for M-ACE requires a great deal of caution in the context of MS. It should be noted that almost half of this result is a derivative of the memory subscale but does not include the language subscale. Taking into consideration these conclusions and the literature reports, this tool is more useful in assessing cognitive functioning in a patient with dementia, where significant disturbances in the field of memory processes are observed [1]. Cognitive dysfunctions in the course of MS, especially at the initial stage of the disease, may not reach significant intensity. High sensitivity and specificity for the M-ACE result has been already described with the loss of one point with a relatively low maximum score (30 points). Inference based on M-ACE result in patients with MS may have limited value and requires interpretation in the context of results of subscales for individual cognitive functions as well as in the light of the results of an extended neuropsychological examination. Despite the unquestionable usefulness of the ACE-III test, it requires further research to normalise it for Polish population.

CONCLUSIONS

1. The ACE-III test may be an extended screening test useful for assessing the cognitive functioning of MS patients.
2. The ACE-III test shows high diagnostic accuracy as well as good sensitivity and diagnostic specificity in the assessment of cognitive dysfunctions in the course of MS.
3. The study is an attempt to determine the cut-off points for individual cognitive functions, the ACE-III total score and the result of M-ACE.
4. The results obtained in the study with the ACE-III test concerning the deteriorated functioning of MS patients in terms of memory and verbal fluency coincide with the conclusions from the literature on cognitive dysfunctions in MS and are an attempt to characterise the profile of cognitive dysfunction presented by patients with MS.
5. The ACE-III test should be supplemented with tools to assess mood disorders.

Conflict of interest/Konflikt interesu

Absent./Nie występuje.

Financial support/Finansowanie

Absent./Nie występuje.

Authors’ contribution/Wkład pracy autorów

Małgorzata Figlus and Magdalena Obrembska contributed equally to this work./Małgorzata Figlus i Magdalena Obrembska deklarują równy wkład w niniejszą pracę.

References/Piśmiennictwo

1. Guenter W, Jabłońska J, Bieliński M, Borkowska A. Neuroobrazowe i genetyczne korelaty zaburzeń funkcji poznawczych w przebiegu stwardnienia rozsianego. Psychiatr Pol 2015; 49: 897-910.
2. Mathuranath PS, Nestor PJ, Berrios GE, Rakowicz W, Hodges JR. A brief cognitive test battery to differentiate Alzheimer’s disease and frontotemporal dementia. Neurology 2000; 55: 1613-1620.
3. Matias-Guiu JA, Cortés-Martínez A, Valles-Salgado M, et al. Addenbrooke’s cognitive examination III: diagnostic utility for mild cognitive impairment and dementia and correlation with standardized neuropsychological tests. Int Psychogeriatr 2017; 29: 105-113.
4. Hsieh S, McGrory S, Leslie F, et al. The Mini-Addenbrooke’s Cognitive Examination: a new assessment tool for dementia. Dement Geriatr Cogn Disord 2015; 39: 1-11.
5. Hsieh S, Schubert S, Hoon C, Mioshi E, Hodges JR. Validation of the Addenbrooke’s Cognitive Examination III in frontotemporal dementia and Alzheimer’s disease. Dement Geriatr Cogn Disord 2013; 36: 242-250.
6. Pendlebury ST, Mariz J, Bull L, Mehta Z, Rothwell PM. MoCA, ACE-R, and MMSE versus the National Institute of Neurological Disorders and Stroke – Canadian Stroke Network vascular cognitive impairment harmonization standards neuropsychological battery after TIA and stroke. Stroke 2012; 43: 464-469.
7. Velayudhan L, Ryu SH, Raczek M, et al. Review of brief cognitive tests for patients with suspected dementia. Int Psychogeriatr 2014; 26: 1247-1262.
8. Giebel CM, Challis D. Sensitivity of the Mini-Mental State Examination, Montreal Cognitive Assessment and the Addenbrooke’s Cognitive Examination III to everyday activity impairments in dementia: an exploratory study. Int J Geriatr Psychiatry 2017; 32: 1085-1093.
9. Matías-Guiu JA, Valles-Salgado M, Rognoni T, Hamre-Gil F, Moreno-Ramos T, Matías-Guiu J. Comparative diagnostic accuracy of the ACE-III, MIS, MMSE, MoCA, and RUDAS for screening of Alzheimer Disease. Dement Geriatr Cogn Disord 2017; 43: 237-246.
10. DeSousa EA, Albert RH, Kalman B. Cognitive impairments in multiple sclerosis: a review. Am J Alzheimers Dis Other Demen 2002; 17: 23-29.
11. Ferreira ML. Cognitive deficits in multiple sclerosis: a systematic review. Arq Neuropsiquiatr 2010; 68: 632-641.
12. Johnen A, Landmeyer NC, Bürkner PC, Wiendl H, Meuth SG, Holling H. Distinct cognitive impairments in different disease courses of multiple sclerosis – A systematic review and meta-analysis. Neurosci Biobehav Rev 2017; 83: 568-578.
13. Rock PL, Roiser JP, Riedel WJ, Blackwell AD. Cognitive impairment in depression: a systematic review and meta-analysis. Psychol Med 2014; 44: 2029-2040.
14. Sitek EJ, Barczak A, Senderecka M. Zastosowanie jakościowej analizy profilu wykonania skali ACE-III w diagnostyce różnicowej chorób otępiennych. Aktualności Neurologiczne 2017; 17: 34-41.
15. Pravatà E, Rocca MA, Valsasina P, et al. Gray matter trophism, cognitive impairment, and depression in patients with multiple sclerosis. Mult Scler J 2017; 23: 1864-1874.
16. Amaral-Carvalho V, Caramelli P. Normative data for healthy middle-aged and elderly performance on the Addenbrooke Cognitive Examination-Revised. Cogn Behav Neurol 2012; 25: 72-76.
17. Singh-Manoux A, Kivimaki M, Glymour MM, et al. Timing of onset of cognitive decline: results from Whitehall II prospective cohort study. BMJ 2012; 344: d7622.
18. Benedict RHB, DeLuca J, Enzinger C, Geurts JJG, Krupp LB, Rao SM. Neuropsychology of Multiple Sclerosis: Looking Back and Moving Forward. J Int Neuropsychol Soc 2017; 23: 832-842.
19. Laws KR, Irvine K, Gale TM. Sex differences in cognitive impairment in Alzheimer’s disease. World J Psychiatry 2016; 6: 54-65.
20. Gałecki P, Talarowska M. Poznanie, emocje, depresja, procesy zapalne – podstawowe założenia teorii neurorozwojowej depresji. Neuropsychiatry and Neuropsychology 2017; 12: 30-37.
21. Lees RA, Hendry Ba K, Broomfield N, Stott D, Larner AJ, Quinn TJ. Cognitive assessment in stroke: feasibility and test