Evaluation of convergent and discriminant validity of an online screening protocol for the detection of mild cognitive impairment in elderly people in Chile.

Nieves Schade Y., Catalina Poblete G., Yaranay Lopez A., Antonio Sanchez C., Lizbeth De la Torres

ABSTRACT


Introduction: Early detection of cognitive impairment in the elderly allows us to reduce uncertainty and improve intervention tasks. The objective of this research was to determine the psychometric characteristics of an online screening protocol for early detection of mild impairment in the elderly. Method: the method used was of a mixed type with quantitative and qualitative questions. The sample consisted of 75 older people from the greater Concepción, Bio-Bio region. The protocol consisted of: Sociodemographic Questionnaire, Cacho Version Clock Test, Mocha, Yesavage Depression Scale and Word Stress Test. The procedure consisted of the application of the protocol through a Tablet or Laptop, video calls. Results: It was found that the protocol presents discriminant and convergent validity. Conclusions: it is concluded that the online screening protocol for early detection of mild deterioration in the elderly is valid and reliable. The use and application of a screening protocol using Teleneuropsychology (TeleNP) in older people is discussed.

Keywords: protocol, mild cognitive impairment older people

INTRODUCTION


In Chile there is a steady increase in the number of elderly people; the current percentage is 16.2% and it is expected that by 2050 they will account for 20% of the Chilean population(1). This increase is already considered a global phenomenon with a progressive “aging of the elderly population”(2).

With aging, the prevalence of Alzheimer’s disease and other types of dementia also increases, especially after the age of 65 years, with women being the most affected(3). An increase of 77% in the number of people with dementia in countries such as Chile is expected by the year 2040(4). Ibáñez et al. point out that the increase in dementias is alarming, which could become a very critical health situation for most countries (5).

With increasing age, older people may be affected in their motor skills, general body and cognitive functioning, which could in turn lead to dementia. Mild cognitive impairment (MCI) is defined as the loss of mental faculties, in which there is a subjective and objective deficit of cognitive functions, specifically impairment of attention and concentration, memory, expressive and comprehensive language, reasoning, judgment, planning, adaptation to new situations, and visuoconstructive and spatial orientation skills (2).

MCI may be the preamble to Alzheimer’s disease, vascular dementia or other secondary diseases that may cause dementia, but in some cases MCI may be the manifestation of some stable or reversible disorder that does not progress to dementia. MCI depends on both physiological and environmental factors and is subject to great interindividual variability(2). Given the complexity of discriminating processes of decline vs. deterioration, it is important to have sensitive and reliable protocols that determine the differences, or the specificity of the outcome of interventions(6).

Early diagnosis is very relevant for the early detection of MCI and possible dementia, to try to palliate symptoms and associated comorbidities and to be able to promptly implement effective treatments (7). In addition, it helps to end uncertainty, confirm suspicions, increase understanding of the problem, promote coping strategies, facilitate personal planning, access treatment and provide access to social support measures (8).

In the manual of good practices for the diagnosis of dementia prepared by Ibáñez et al. (2020) (5), there is concern about the low social awareness of dementia, the absence of standards for cognitive assessments, restricted access to health services, almost no training of teams and lack of data records, stigma and economic inequality in care and attention to these people; which partly explains the difficulties that the countries of Latin America and the Caribbean currently present in the face of dementia. In Chile there is a National Dementia Plan that puts it in a more advanced position in relation to the countries of the region. However, the need to improve the diagnosis of dementia is recognized (9)

This same manual(5) recommends that in order to perform a cognitive screening it is necessary that the assessment be brief, for which it is important to have an overall picture of the cognitive functions of a person in a short period of time. The instrument should be very sensitive in that it can distinguish between cognitive changes resulting from the normal aging process, mild cognitive impairment, dementia and Alzheimer’s disease. In addition, it should be brief, easy to apply and not require extensive training to do so, so that it can be applied by primary care professionals or others (5).

In the field of dementia, neuropsychological assessment has been shown to be useful: the Mini-Mental State Examination (MMSE) and the Montreal Cognitive Assessment (MoCA). The MoCA is a more complex brief battery that assesses a wider range of cognitive domains, presenting high sensitivity for detecting mild impairment and other types of dementia, not only Alzheimer’s disease. Another commonly used neuropsychiatric test is the Yesavage Geriatric Depression Scale, although it is often considered complementary. In general, it is considered that these tests should comprise different cognitive domains or processes and there is currently no agreement on which of them is more effective and sensitive in establishing this diagnosis(10). The complex reality of the transition from the normal decline of aging to the deterioration of the pathological process is complex and highly dependent on the biography of the subject (11).

As a way to improve diagnosis and time, there are screening tests, which constitute a secondary prevention measure that consists of performing diagnostic tests on people who a priori are considered healthy, in order to detect possible pathologies in an early manner. Cognitive screening with brief neuropsychological tests is the method that has contributed most to the early detection of cognitive deficits or pre-dementia states.(12)

According to Carnero-Pardo et. al(13), screening tests must meet the following general characteristics when applied: (a) they must be short, since in Primary Care an average of less than 10 minutes is available to treat each patient; (b) they must be easy, since the professionals who are required to use them are usually neither experts in dementia nor in test application; (c) they must be simple and not require instrumentation that prevents their use in varied and non-specific settings, such as doctors’ offices, homes, etc. (d) they must be inexpensive; (e) they must be acceptable and equitable for all subjects (e.g., illiterate or poorly educated people); (f) therefore, they must be adaptable to a wide cultural and linguistic variability; (g) finally, they must be flexible, i.e., they must be adjusted to the diagnostic difficulty of each case.

Based on the above, a team formed by international researchers from Mexico and Spain: De la Torre and Sánchez-Cabaco (2020) set out to create a screening protocol for the early detection of cognitive impairment and to test it in different countries, including Chile. In addition, the protocol design follows the guidelines of the team led by neurologist Dr. Cacho as a pioneer in the field in the adaptation of the Clock Test and other screening tests, as well as the applications to the clinical diagnostic setting (14).

As pointed out by Gonzalez-Martinez et al. (15) the diagnostic criteria for MCI have not been stable for the identification of this critical phase of the onset of pathological aging. The protocol presented in this paper avoids the initial historical bias of neuropsychological testing toward amnestic-type MCI and also avoids some keys to the single-multiple domain debate.

The aim of this research was to conduct a cross-sectional descriptive psychometric study of an online screening protocol for early detection of mild impairment in the elderly. An exploratory study of the application of this protocol had previously been carried out during the COVID-19 pandemic, the aim of which was to determine subjective aspects such as the degree of collaboration, motivation and ease of understanding instructions and application time in a teleneuropsychological consultation (16).

It was found that most of the tests presented a high level of reliability and the elderly presented very good motivation to respond, ease and a short application time, although some difficulties in handling the technology required support (17).

The first specific objectives of this study were to determine criterion validity, which consists of establishing a link between performance on the tests studied and performance on some other criterion (18).

One of the approaches to establish criterion validity is to compare the score of the test evaluated with another test (19), (20). Criterion validity comprises convergent validity and discriminant validity. The former is understood as the positive correlation between the tests considered, which would indicate that both tests comprise the same process. And the second is considered the opposite process, i.e., the absence of correlations between the tests, therefore, the tests in question do not comprise the same processes(21). Convergent/divergent validity was evaluated based on the correlation coefficients between the scores of the dimensions of the three instruments.

The second objective was to verify the internal consistency of each instrument considering the target population.

METHOD


Sample

The accidental sample consisted of a total of 75 elderly people from Gran Concepción, mainly women and aged between 60 and 80 years, with an average age of 68 year

Instruments

MoCA-S: Montreal Cognitive Assessment Scale MoCA(21), but in its Spanish version, evaluates executive and visuospatial function, identification, memory, attention, language, abstraction, memory and orientation; with an application time of 5 to 10 minutes. The specificity of evaluating the psychometric properties and the discriminative validity of the MoCA-S has been estimated as excellent (90%) for detecting mild cognitive impairment. The MoCA-S in the elderly has the reliability and validity to be used in Chile (22).

Word Accentuation Test TAP: allows the estimation of previous intelligence and with diagnostic capacity in dementia. The Spanish adaptation of the TAP shows a correlation with the WAIS and with other general intelligence tests above 0.80, much higher than with the Mini mental (MMSE) (23), (24).

Cacho Version Clock Test (25) TR: is a non-intrusive instrument for the patient and safe to use for the identification of mild Alzheimer’s dementia, because it evaluates among other functions the visuospatial deficit, which is an early sign of dementia. This TR is valid, brief and easy to apply in the Chilean population (26).

Yesavage Depression Scale Reduced Version: 5-item abbreviated Geriatric Depression Scale (5-GDS) assesses depression developed by Brink and Yesavage 3019 yes-no questions. Sheikh and Yesavage developed the abbreviated 15-item version, which retains the effectiveness of the original scale, improving the ease of administration. Studies in Chile show that it is quite effective as a screening instrument, with good sensitivity, specificity and positive and negative predictive values (27).

Procedure

The application of the tests was through video calls either using laptop or cell phone according to the consensus indications in the area of neuropsychological assessment mediated by ICT, in the new discipline called teleneuropsychology (TeleNP) (16).

The application procedure was as follows: first each participant was asked to read and sign the informed consent form and then the instruments were applied.

Ethical aspects. Participants were informed of the purpose, risks and benefits of the study by means of informed consent. The project was approved by the Ethics Committee of the Universidad de Concepción, within the context of a larger University project.

RESULTS


Three types of analysis are presented below, the first one has to do with a descriptive analysis of all the general characteristics of the elderly under study. At secondly, an analysis of variance and correlation between the different instruments was carried out to determine whether there were significant differences and relationships between the instruments that make up the protocol, and to determine whether it has discriminant or convergent validity and degree of reliability.

1.Descriptive statistics

The group of elderly people presented mild cognitive impairment. In addition, most of them had incomplete education, read 2 or 3 times a week, had heart problems and consumed medication, did very little physical activity and had very high social participation.

2. Analysis of the instruments

The following instruments were applied: the Yesavage Depression Scale, MoCA, Cacho Clock Test, TAP Word Accentuation Test. The Yesavage Depression Scale was considered in a complementary manner, only the first 3 were included in the analysis.

2.1Yesavage Depression Scale

It was found that 72.7% of the persons were mildly depressed.

2.2MoCA (Montreal Cognitive Assessment)

With the MoCA (Montreal Cognitive Assessment) instrument, it was found that most of the sample showed cognitive impairment (61.3%)

It is also important to point out that in this application the instrument reached an average of 0.68 in Cronbach’s Alpha. Its item-test correlation decreases in question 10.

We evaluated whether there were differences between the MoCA indicators and the characteristics of the individuals, and found a clear inverse relationship with age, i.e. the older the person is, the greater the probability of presenting cognitive deterioration r = -0.61. Those who performed better on the scale had complete schooling t (72.81)= 7. 0253, p< 0.005 performed more physical exercise F(73)= 5.969 p<0.00109, read more F (70) 2.576 p > 0.0449, slept approximately 6 to 7 hours F (71)= 2.837 p>0441, and had more social participation F(73)= 32.05 p< 000.1.

No differences were found by sex. There is a significant relationship between having illness F(73 ) 5.893 p>0.0177 and having higher indicators of cognitive impairment.

2.3 Cacho Clock Test Version

In relation to the Cacho Version Clock Test et. al. instrument, it was recorded with this instrument that people had a higher percentage of cognitive impairment. An average Cronbach’s alpha of 0.717% was found.

2.4 Word Accentuation Test (TAP)

This test shows that older people had a significant percentage of cognitive impairment. A Cronbach’s alpha of 0.843% was found.

3. Analysis of Correlations and Variance between instruments

The objective was to determine the discriminant and convergent validity between the instruments that measure cognitive impairment, for which analysis of variance and correlations between them was performed.

It was found that there was a strong correlation between the MoCA and the Cacho Clock test r = 0.68, see Figure 1.

Similar results were found with the Word Accentuation Test TAP, with a correlation of r= 0.71 with the MoCA. In addition, it was found that the correlation between this TAP and the clock was r= 0.61. See Graph 2.

Considering the shortness of the clock and word stress test, a prediction model of the adjusted total MoCA was tested. Both variables were found to be significant.

Finally, a study was made to know the normality and homogeneity distribution. The normality assumption and the homoscedasticity assumption were tested, with Levene’s statistic in the variance, it was found that these were normal F(72) =51.51 p< 0.001.

Finally, the predictive ability of the three instruments was analyzed and it was found that, when rounded to one decimal place, with 21.5 or less in the predicted value, all persons were found to be impaired. On the other hand, if it is increased to 28 points or more, all people appear normal.

Discussion

The aim of this research was to conduct a psychometric study of an online screening protocol for early detection of MCI in Chilean elderly people. It was found that the protocol presents a positive correlation between all its tests, especially the MoCA with the TAP and then with the Clock Test. A lower positive correlation was observed between the TAP and the Clock Test. From these positive correlations it is evident that there is convergent validity of the protocol. Likewise, internal consistency was high in all tests.

The protocol screened for MCI is found to have valid and reliable characteristics for older people in Chile. In a previous work, with the same testing protocol, carried out with a population of people over 60 years of age in Mexico (28), the results of the statistical analysis suggest a moderate correlation between the level of autonomy and cognitive performance and a good correlation of cognitive reserve with cognitive impairment and the educational level of the participants.

It was found that a significant group of people presented mild cognitive impairment, had a history of pathology, took medication, did not engage in any physical activity, read little, slept between 6 and 7 hours and participated socially in the study, and had no physical activity.

These results are consistent in terms of the relationship between mild cognitive impairment and the aforementioned lifestyles. Recent evidence suggests the need for a broad approach to the construct of cognitive reserve, linked to aspects of physical activity and the meaning of experiences or life project, which refers to emotional and motivational reserve (29).

The above results should be contrasted with a larger population group in order to confirm the findings of this protocol, and thus be able to detect and diagnose MCI.

In future work, and with the indicated sample expansion, differential profiles by sex should be established since it has been found (30) that women show worse cognitive performances in domains not related to memory, and the analysis of learning curves showed additional advantages specific to the male sex that are not detected by MCI subtypes.

Finally, with the increase in the population, early diagnosis is necessary through valid and reliable psychometric systems, easy to apply in order to perform remedial actions as soon as possible and thus prevent some type of dementia and improve the quality of life of the elderly. In the field of ICT’s applications are scarce and with little scientific evidence, so it is necessary to advance in the academic support on its use in the evaluation (31), since in the intervention scenario has developed significantly in recent years (32).

REFERENCES


  1. 1. National Institute of Statistics. Chili. Older Adults in Figures. Vital Statistics 2016. INE, 2018. Available at URL: https://www.ine.cl/docs/default-source/ ublicaciones/2018/adultos-mayores-cifras-vitales.pdf?sfvrsn=1eff56d2_6
  2. 2. Benavides-Caro C. Cognitive impairment in the elderly. Journal of Anesthesiology. 2017; 40(2):107-112.
  3. 3. Parra, M. A., Báez, S., Allegri, R., Nitrini, R., Lopera, F., Slachevsky, A., et al., . Dementia in Latin America: Assessing the present and envisioning the future. Neurology. 2018; 90(5), 222-231.
  4. 4. WHO Clinical Consortium on Healthy Aging. Topic focus: frailty and intrinsic capacity. Report of consortium meeting December 2016 in Geneva, Switzerland, 2017. Available at URL: https://apps.who.int/iris/bitstream/handle/10665/272437/WHO-FWC-ALC-17.2-eng.pdf?ua =1
  5. 5. Ibáñez A., Slachevsky A, Serrano C. Manual of Good Practices for the diagnosis of dementia. SENAMA. (2020) Govt. From Chile. Available at URL:https://www.senama.gob.cl/storage/docs/Ibanez-ASlachevsky-A-Serrano-C.-Manual-de-Buenas-Practicas-para-el-diagnostico-dedemencia.pdf
  6. 6. Borrás C., Viña J. Neurophysiology and aging. Concept and pathophysiological bases of cognitive impairment Spanish Journal of Geriatrics and Gerontology. 2016; (56)1: 3-6.
  7. 7. MINSAL. National Mental Health Plan 2017-2015. Ministry of Health. Chili. 2017 available at URL: https://www.minsal.cl/wp-content/uploads/2017/12/PDF-PLAN-NACIONAL-SALUD-MENTAL-2017-A-2025.-7-dic-2017.
  8. 8. Loureiro C., García C, Adana L., Yacelga, T Rodríguez-Lorenzana A., Maruta, C. Use of the Montreal Cognitive Assessment Test (MoCA) in Latin America: systematic review. Rev Neurol. 2018; (66): 397-408.
  9. 9. Rodríguez L, Saracco-Álvarez R, Escamilla-Orosco R, Fresán A. Validity of the Montreal Cognitive Assessment Scale (MoCA) to determine cognitive impairment in patients with schizophrenia. Mental Health 2014; (37): 517-22.
  10. 10. Meléndez, J. C. M., Sanz-Álvarez, T., & Navarro-Pardo, E. Mild cognitive impairment: classification method and procedure. Annals of Psychology. 2012; (28) 2: 604-610.
  11. 11. Martin E, Velayudhan L. Neuropsychiatric Symptoms in Mild Cognitive Impairment: A Literature Review. Dement Geriatr Cogn Disord. 2020; 49(2):146-155.
  12. 12. Loureiro C., García C, Adana L., Yacelga, T Rodríguez-Lorenzana A. and Maruta, C. (2018). Use of the Montreal Cognitive Assessment Test (MoCA) in Latin America: systematic review Rev Neurol. 2018; 66 (12): 397-408.
  13. 13. Carnero-Pardo C, Gurpegui M, Sánchez-Cantalejo E, Frank A, Mola S, Barquero MS, et al. Diagnostic accuracy of the Eurotest for dementia: a naturalistic, multicenter phase II study. BMC Neurology. 2006: 6-15
  14. 14. García-García-Patino, R., Benito-León, J., Mitchell, A. J., Pastorino-Mellado, D., García García, R., Ladera-Fernández, et al. Memory and executive dysfunction predict complex activities of daily living impairment in amnestic multi-domain mild cognitive impairment. Am. J. Alzheimer’s Dis. 2020; 75(3): 1061–1069.
  15. 15. González-Martínez P., Oltra-Cucarella J., Sitges-Maciá E., Bonete-López B. Review and update of the objective cognitive impairment criteria. Rev. Neurol, 2021; (72): 288-295. https://doi: 10.33588/rn.7208.2020626.
  16. 16. Pérez P., Ramos D., Arango J.C. Teleneuropsychology in Spanish-speaking countries: A critical look at the use of Information and Communication Technologies in neuropsychological evaluation. Rev. Iberoam. from Psychol. and health. 2021; (4)1: 1-27.
  17. 17. Schade N., Medina F., Ramírez-Vielma R., Sánchez-Cabaco A., De La Torres L. Early detection of Mild Cognitive Impairment in older people during the pandemic: online screening protocol. Rev. Chil Neuro-Psychiat, 2022; 60 (4); 403-412.
  18. 18. American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and Psychological Testing (1st). Washington, DC: American Educational Research Association.
  19. 19. Carretero-Dios, H., & Pérez, C. Standards for the development and review of instrumental studies: Considerations about test selection in psychological research. Int. J. Clin. Health Psychol. 2007; 7(3): 863-882.
  20. 20. Hogan, T. (2004). Psychological Tests A practical introduction (1st). Mexico City: The Modern Manual.
  21. 21. Olazarán, M.C. Hoyos-Alonso, T. Del Ser, A. Garrido-Barral, J.L. Conde-Sala, F. Bermejo-Pareja, S. et. to the. Practical application of brief cognitive tests. Rev Neurol. 2016; 31(3): 183-194.
  22. 22. Delgado, Aravena & Behrens. Validation of the Montreal Cognitive Assessment instrument in Spanish in adults over 60 years of age. Rev Neurol. 2019; 34 (6): 376-385
  23. 23. Del Ser, T., González, J., Martínez, S., Delgado C., & Bermejo, F. Estimation of premorbid intelligence in Spanish people with the Word Accentuation Test and its applications to the diagnosis of dementia. Brain Cogn. 1997; 33: 343-356. https://psycnet.apa.org/doi/10.1006/brcg.1997.0877
  24. 24. Sierra N., Torrealba T., Roca M., Manes F., Burin D. Estimation of premorbid intelligence in mild and moderate cognitive impairment and executive deficit. Rev. Iberoam. neuropsychol. 2010; (2)1: 25-32.
  25. 25. Cacho J., Benito-León J., García-García R., Fernández-Calvo B., Vicente-Villard J.L., Does Mitchell A. Does the Combination of the MMSE and Clock Drawing Test (MiniClock) Improve the Detection of Mild Alzheimer's Disease and Mild Cognitive Impairment? Am. J. Alzheimer’s Dis. 2010; 22: 889–896.
  26. 26. López N., Allegri R., Soto M. Diagnostic capacity and preliminary validation of the clock test, on-demand version, for mild Alzheimer's disease in the Chilean population. Ecuadorian Rev. of Neurol. 2014, 23: 1-3.
  27. 27. Hoyl T, Valenzuela E., and Marín, P. Depression in older adults: preliminary evaluation of the effectiveness, as a screening instrument, of the 5-item version of the Geriatric Depression Scale. Rev Med. Chil. 2000; (128)11: 1199-1204.
  28. 28. Cabaco, A.S, De la Torre, L., Álvarez, D., Mejía, M. & Wobbeking, M. Tele neuropsychological exploratory assessment of indicators of mild cognitive pairment and autonomy level in Mexican population over 60 years old. PEC Innovation, 2023; 2(100107): 1-7.
  29. 29. Cabaco, AS, Wöbbeking, M., Mejía-Ramírez, M, Urchaga-Litago, JD, Castillo-Riedel, E and Bonete-López, B. (2023). Mediation effects of cognitive, physical, and motivational reserves on cognitive performance in older people. Front. Psychol. 13, 1112308.
  30. 30. Karstens, A., Maynard, T., & Tremont, G. Sex-specific differences in neuropsychological profiles of mild cognitive impairment in a hospital-based clinical sample. J Int Neuropsychol Soc. 2023; 1-10.
  31. 31. Cubillo-Leivas, A. M., Olivares-Olivares, P. J., & Rosa-Alcázar, A. Telepsychological Mobile Applications in the Spanish Android Market. Psychothema. 2022; 34(4), 498-507
  32. 32. Sitges-Maciá, E., Bonete-López, B.; Sánchez-Cabaco, A., Oltra-Cucarella, J. Effects of e-Health Training and Social Support Interventions for Informal Caregivers of People with Dementia. A Narrative Review. Int. J. Environ. Public Health Res. 2021; 18(7728), 1-16.


HOW TO QUOTE?


(2024). Evaluation of convergent and discriminant validity of an online screening protocol for the detection of mild cognitive impairment in elderly people in Chile..Journal of Neuroeuropsychiatry, 57(4).
Recovered from https://www.journalofneuropsychiatry.cl/articulo.php?id= 166
2024. « Evaluation of convergent and discriminant validity of an online screening protocol for the detection of mild cognitive impairment in elderly people in Chile.» Journal of Neuroeuropsychiatry, 57(4). https://www.journalofneuropsychiatry.cl/articulo.php?id= 166
(2024). « Evaluation of convergent and discriminant validity of an online screening protocol for the detection of mild cognitive impairment in elderly people in Chile. ». Journal of Neuroeuropsychiatry, 57(4). Available in: https://www.journalofneuropsychiatry.cl/articulo.php?id= 166 ( Accessed: 14junio2024 )
Journal Of Neuropsichiatry of Chile [Internet]. [cited 2024-06-14]; Available from: https://www.journalofneuropsychiatry.cl/articulo.php?id=166

 

DOWNLOAD PDF VERSION