Translation of the Alberta Continuity of Services Scale for Mental Health to a Latin American context

ABSTRACT

Objective: Describe the process of translation, cultural adaptation to Colombia, as well as the internal consistency and construct validity of the Alberta Continuity of Services Scale for Mental Health (ACSS-MH). Methods: This instrument was subdued to the evaluation of validity of the content by experts and this was applied to a rural population in a Colombian context. Were performed tests of internal consistency and construct validity for each of the parts of the scale. Results: Under the consensus of the expert, it is made changes on some items, looking for a better adaptability of the instrument to the linguistic characteristics of Spanish, without losing sight of the evaluation objective of each one of the items on the original questionnaire. The result of the analysis of part A converged in 5 components that explain the 69.69% of the variance with 24 Items; Similarly, the analysis of part B grouped 13 items into four components, which explain the 72.02% of the variance. Discussion: This scale could be implemented to improve the provision of mental health services in Latin American contexts, where continuity of care has presented significant difficulties.

Keywords: research/service delivery; primary care; mental health systems/hospitals; patient needs; quality of care.

INTRODUCTION

The World Health Organization(1) on the Global Health report, “Mental Health: New Understandings, New Hope”, reviewed the needs of the mental health on relation to the public policies, the public health and the mental health services. In other aspects, it prompted to integrate the treatment of the neuropsychiatric disorders in the primary attention and to integrate the psychiatric hospitals in general hospitals. In 2005, the Brasilia Principles, which reiterate the validity of the Caracas Declaration in relation to the protection of human rights and fundamentals freedoms of the affected people by mental disorders, emphasize the need to establish community services that replace the psychiatric hospital and ensure provision of an adequate integrate and multidisciplinary attention and care of people with psychiatric disorders and with crisis situations, including, when necessary, the admission to general hospitals, the participation of users and relatives on the planning and development of programs and mental health services, and agreement of actions with the different social actors in order to improve the mental health status of the population(2). Despite the mentioned above, it has been possible to reflect that between 75-90% of people with mental disorders do not receive the necessary treatment despite the fact that there is an effective treatment(3).

Continuity of care is one of the central elements in public international health systems that have placed their emphasis on the community. Especially in mental health, it has become a standard for interventions as it is a central element in their quality and because it has been proven that the effectiveness of continuity of care programs is superior to interventions that do not include it(4,5,6). Although the use of language to describe or define the concept of continuity of care is not always consistent, there is a theoretical consensus of it, stating that future research should focus on understanding it and operational measurement of it(7), in this sense, continuity of care is considered as a process that involves an orderly care and attention, an uninterrupted movement of people among several elements of the provision service system(8), this factor is essential to obtain positive results for people with serious and persistent mental illnesses(9) since there are multiple associations between the continuity of mental health services and the sustained quality of life in multivariate models(10).

In relation to the mentioned above, although it is documented the relation between the continuity of services of mental health and the quality of life of the users, according to Díaz-Castro et al., it has been found(11) in Latin American countries some problems in the management of public health related to mental health. Among these, are notable the scarce of research resources, the functional fragmentation of mental health systems, the absence of a national health system, the lack of policies that directly affect the organization and the provision of services, or policies that are poorly managed. Similarly, it has been described a higher spending on psychiatric hospitalizations in countries that allocate fewer resources for mental health, showing that countries have fewer psychiatric hospitalizations while the greater is the investment in community-based services(12,13).

Despite the recognition of the importance of continuity of care in mental health and the need to systematize the evaluations of programs(14), it does not exist enough evidence in terms of measurement instruments of this continuity in Latin America. Even globally it has been conceptualized that this matter has been abandoned repeatedly by the research teams, possibly due to the shortage of validated multidimensional measures and to the lack of studies designed to approach the inherent complexities of the service(15,16). Although indicators related to coverage, citizen participation in health and differentiation in hospital and outpatient settings have been developed, it is still not very clear how to determine the continuity of care services with each other (hospital to outpatient processes, outpatient processes to community processes, support in community processes), therefore, it is essential to measure the implementation of health programs, plans, and strategies, based on continuity of care (Figure 1). The objective of this work is to describe the process of translation, cultural adaptation to Colombia, as well as the internal consistency and construct validity of the ACSS-MH.

METHODS

A descriptive cross-sectional study was made to evaluate the psychometric characteristics of the Alberta Continuity of Services Scale-Mental Health, ACSS-MH in order to adopt this instrument to being utilized by the users of mental health services in the Colombian context(17,18).

Instrument

It was used the ACSS-MH. This instrument is divided into two parts, the first (Part A) aimed at users and made up of 43 Likert-type items that inquire about the use of mental health services and the subjective experience regarding the process of providing care through 3 subscales (Fragmentation of the system, base relations and receptive treatment), and the second one (Part B) made up of 17 multiple-choice items with a single answer through which the provision of services is reported from the records of the care process.

In the process of translation and validation of judges, the methodology used was the one proposed by Bracken and Barona, who suggest(19) guidelines for the translation of tests, which began with the choice of the test, the simplification of its items in relation to a cultural context, the translation of the test by a translation team, the revision of the translation by experts in mental health and by the author of the scale, with their respective recommendations in relation to the items and the implementation of the scale(20).

The Scale was translated by members of the research team, to later develop, by consensus, a preliminary version of the instrument into Spanish. Subsequently, the differences were discussed (use of synonyms, terms rarely used in everyday language, different structures in sentences or complex language) and a final version was built, according to Gjersing et al(21). This version was then subjected to a reverse translation (Spanish-English) by a professional translator, who also translated the instrument into the original language (English-Spanish). Both translations were compared with the original instrument and no major semantic differences were found. These versions were reviewed by the main author of the Scale (Carol Adair), who, after some adjustments, endorsed both translations.

To carry out the content validity, the opinions of five experts in child and adolescent mental health, in research and teaching were collected (a clinical psychologist, PhD in psychology; a psychologist, with a master in community interventions; a psychologist, PhD in social, childhood and youth; a psychologist, PhD in psychology and with a master in public health; a psychiatrist, specialist in epidemiology), based on the methodology proposed by Escobar-Pérez and Cuervo-Martínez(22).

Through a digital form, each expert evaluated the 43 items of part A and the 17 items of part B of the instrument. For the elaboration of the virtual tool, 10 categories were considered on the Items: clarity, objectivity, relevance, organization, sufficiency, adequacy, consistency, coherence, methodology, significance. The V Aiken (Coefficient of Content Validity) was applied to these items(23), considering the adequate item above.

In addition, this instrument was subdued to the evaluation of validity of the content by experts, based on the parameters of Lawshe(24). Through the Content Validity Index (CVI): it was assigned to each one, one of the following categories: unnecessary, useful, or essential. Besides, it was inquired whether each question was understandable.

The CVI calculation for each item was performed according to the following formula:

CVI = (ne - N/2)/N/2

Where ne is the number of experts who have assessed the item as essential and N is the total number of experts who have assessed the item. The CVI oscillates between +1 and -1, where the positive scores are the ones that indicate better content validity. Items with a range between 50 and 99 were considered adequate. For the translation validity analysis, it was considered whether the operationalization in items was an adequate reflection of the construct, considering the knowledge of the researchers about these items and also, about the functioning of the instrument in a general view(25). After the translation validity analysis, it can be continued with the criteria validity analysis(26).

Once the translated version of the Scale was available (Appendix), it was applied. For the application of the instrument, were selected people who consulted in the first level of care with diagnoses of chronic mental disorders, in semi-urban and rural areas of the department of Caldas, Colombia. The majority of the population had a low socioeconomic status, with a low level of schooling, although all of them understood the instrument. Part B was completed by doctors and nurses from the primary care level. Both processes were conducted in Spanish.

The instrument was applied to 256 individuals with mental pathology and their families, by health professionals at the first level of care, especially psychology and nursing. Part B was applied by doctors (46.87%), psychologists (31.25%) and nurses (21.87%) from the first level of care, from the institutions where the patients who filled out part A were attended. All of them signed the informed consent.

Once the data was arranged in an Excel file, the information was refined by eliminating repeated cases and replacing the missing data (less than 20%) with the median of the item to which the missing data belonged(27). Then, using SPSS (version 26), were performed tests of internal consistency (Cronbach’s Alpha) and construct validity (varimax rotation factor analysis) for each of the parts of the scale(28).

The project was presented to the bioethics committee of the University of Manizales, which was endorsed by the document CBE02_2022.

RESULTS

Table I shows the assessment by experts of the scale in general.

It is found that questions 15, 20 and 36 were assigned a lower CVI, by part A of the scale. In part B, the Items were not lower than 50. Regarding the translation process, question 15 was translated as Can I go to the main health staff if I am sick or if I am feeling well. Considering the mentioned qualification and that the questionnaire evaluates aspects related to the continuity of mental health services, the question was changed to Can I go to the mental health personnel assigned as the main one with symptoms or without them, which helps to specify the assignment of a professional as part of the care at the low complexity level of mental health care, and besides, it also strengthen the participation in mental health programs that tend to the recovery and maintenance of the health, beyond the disease, and not only the care in the midst of acute symptoms.

Regarding questions 20 and 36, translated as Have the principal health staff called to find out how I am doing, and Do they remind me of appointments or do they call me if I miss the appointments, respectively, and taking into account the score allocation, they were changed to the questions Does the mental health staff or personnel assigned as principal communicates with me to know how I am doing, in the case of question 20; and Are they awaiting of the scheduling and of my attendance at appointments, in the case of question 36.

The mentioned above validates the greater management capacity of people and their families in the continuity of this care, in addition to considering that telephone calls in some contexts such as Latin America are not possible, and there are other forms of communication.

This was done after delving into the basic articles of the instrument(10,15,29) to better understand the meaning of these questions. Regarding questions 15 and 20, in the original questionnaire they include the concept of principal caregiver, which is defined as the only person that the patient sees most frequently to receive mental health care. This person can be a therapist, a psychiatrist, or a social worker, but is always the person that the patient sees most regularly. This expression could be confused in some Latin American contexts, by

“…trained medical personnel, nurses and other health personnel or staff, (in addition to) parents, spouses or other family members, friends, members of the clergy, teachers, social workers”(30), so it is changed to mental health personnel or staff assigned as principal, which could be more in line with most health systems in this context.

Other comments made by the author consisted of changing the word “care” for health care: “because it is more comparable in the world with the word treatment”. Likewise, she suggested changing the expression “controlled treatment” to “monitored treatment”.

Internal consistency and construct validity

Part A of the scale showed high reliability (Alpha= .93-.93), while part B presented medium-low reliability (Alpha= .67-.73). Particularly noteworthy are the values of the statistical estimation of the item-test relationship, according to which Items 4, 10 and 29 do not meet the minimum values and it puts in consideration their inclusion in the scale (Table 2).

From the analysis of the results of the internal consistency test and the Item-scale correlations, the items with the least relationship and the greatest impact on the alpha value were eliminated. With these data, it was made an analysis of the principal components with varimax rotation, considering items with factor loads greater than 50, communalities greater than 35, and the conformation of factors with a minimum of three items. The result of the analysis of part A converged in 5 components that explain the 69.69% of the variance with 24 Items; similarly, the analysis of part B grouped 13 items into four components, which explain the 72.02% of the variance (Table 3).

DISCUSSION

The objective of this study was the process of translation and content validity analysis of the Alberta Service Continuity Scale in Mental Health. The scale includes scales rated by both the patient and the observer, who could be a researcher or a member of the healthcare staff. It also has an evaluation of independent objective continuity and not dependent of the memory. The patient-rated scale has 37 items rated on 5-point response scales across three subscales: system fragmentation (perceived discontinuity between services), relationship basis (perceived importance of a consistent and reliable relationship with the primary caregiver and the treatment team), and sensitive treatment (characterized as the patient’s experience of specific service actions in response to their needs), having a total possible score of 185; in addition to a part B of 17 items, observer version(15), this instrument provides elements to improve mental health care(29) and to invite health systems to be more compatible in their structure and monitoring(31). Both at the level of part A and part B of the scale, it is found that the associations at the level of internal consistency are very encouraging for the part of the existing evidence in the region, where there are studies with insufficient samples or perceptions from a single point of view, without considering other observers of the health system(32).

Other investigations have related the continuity of the service with the quality of life(33) and with a decrease in symptoms(34), which leads to a reduction in the costs of the health system through fewer hospitalizations, which ultimately impacts on a greater functioning of people, apart from the reduction in mortality(35).

The present sample was drawn from patients who were in the process of being followed up from a first level of care, which makes it possible to point out the difficulties that people with mental problems could encounter on a daily basis in order to recover spaces of daily life beyond the dynamics of the recurrent hospitalizations(36).

It is considered that instruments that determine the continuity of services can contribute to generating implementation variables such as utility, feasibility, relevance, cost-benefit of an intervention for mental health, in the same way, studies are required in our context that relate continuity of care with variables such as quality of life, costs of care processes, number of hospitalizations and participation in community mental health models focused on the first level of care(37). To accomplish the mentioned above, this scale could provide methodological elements within the analysis of these implementation processes.

A strength of this study is obtaining answers from academics who also have experience in the care and management of mental disorders. The incorporation of experienced experts is a highly recommended strategy in research and knowledge transfer in the field of mental health and psychiatry(38). Although the expert consultation included only one round of comments, the methodological decisions of the research team were guided by meetings between the primary authors of the questionnaire, facilitating consensus on the adaptations to be made, which, in this case, were minimal.

In conclusion, it is found that this version of the Alberta Continuity of Services Scale for Mental Health reports acceptable construct validity and high reliability (especially in part A) to be used in some Colombian contexts. It is recommended to consolidate the validity of the construct using confirmatory factor analysis techniques that allow estimating the contribution of an item to one or several dimensions simultaneously, as well as the estimation of possible correlations between the parts of the Scale. Future studies could relate continuity to quality of life, psychosocial disability, and cost of care, in addition to considering new adaptations with an ethnic and intersectional approach.

REFERENCES

1. 1. World Health Organization (WHO). The World health report: 2001. Mental health: new understanding, newhope, 2001. https://apps.who.int/iris/handle/10665/42390.
2. 2. World Health Organization. WHO. Informe de la Reunio?n Internacional sobre la Salud en Todas las Poli?ticas, Adelaida, 2010. Disponible en https://www.who.int/social_determinants/spanish_adelaide_statement_for_web.pdf)
3. 3. Diez-Canseco F, Rojas-Vargas J, Toyama M, Mendoza M, Cavero V, Maldonado H, Caballero J, Cutipé Y. Estudio cualitativo sobre la implementación del Programa de continuidad de cuidados y rehabilitación para personas con trastornos mentales graves en el Perú. Rev Panam Salud Publica. 2020 Dec 14;44:e134. Spanish. doi: 10.26633/RPSP.2020.134. PMID: 33337443; PMCID: PMC7737645.
4. 4. Vandyk AD, VanDenkerkhof EG, Graham ID, Harrison MB. Using the Alberta Continuity of Services Observer-Rated Scale to Measure Continuity of Care in a Psychiatric Population. J Nurs Meas. 2016;24(1):131-46. doi: 10.1891/1061-3749.24.1.131. PMID: 27103249.
5. 5. Alonso Suárez M, Bravo-Ortiz MF, Fernández-Liria A, González-Juárez C. Effectiveness of continuity-of-care programs to reduce time in hospital in persons with schizophrenia. Epidemiol Psychiatr Sci. 2011 Mar;20(1):65-72. doi: 10.1017/s2045796011000138. PMID: 21657117.
6. 6. Digel Vandyk A, Graham ID, VanDenKerkhof EG, Ross-White A, Harrison MB. Towards a conceptual consensus of continuity in mental healthcare: focused literature search and theory analysis. Int J Evid Based Healthc. 2013 Jun;11(2):94-109. doi: 10.1111/1744-1609.12024. PMID: 23750572.
7. 7. Bachrach LL. Continuity of care for chronic mental patients: a conceptual analysis. Am J Psychiatry. 1981 Nov;138(11):1449-56. doi: 10.1176/ajp.138.11.1449. PMID: 7294213.
8. 8. Haggerty JL, Reid RJ, Freeman GK, Starfield BH, Adair CE, McKendry R. Continuity of care: a multidisciplinary review. BMJ. 2003 Nov 22;327(7425):1219-21. doi: 10.1136/bmj.327.7425.1219. PMID: 14630762; PMCID: PMC274066.
9. 9. Mitton CR, Adair CE, McDougall GM, Marcoux G. Continuity of care and health care costs among persons with severe mental illness. Psychiatr Serv. 2005 Sep;56(9):1070-6. doi: 10.1176/appi.ps.56.9.1070. PMID: 16148319.
10. 10. Díaz-Castro L, Arredondo A, Pelcastre-Villafuerte BE, Hufty M. Governance and mental health: contributions for public policy approach. Rev Saude Publica. 2017 Jan 30;51:4. doi: 10.1590/S1518-8787.2017051006991. PMID: 28146159; PMCID: PMC5286910.
11. 11. Agudelo-Hernández F., Rojas-Andrade R. Ciencias de la Implementación y Salud Mental: un Diálogo Urgente. Revista Colombiana de Psiquiatría, 2021. ISSN 0034-7450. https://doi.org/10.1016/j.rcp.2021.08.001.
12. 12. Agudelo-Hernández F, Rojas-Andrade R. Mental Health Services in Colombia: A National Implementation Study. Int J Soc Determinants Health Health Serv. 2023 Mar 6:27551938231156023. https://doi.org/10.1177/27551938231156023.
13. 13. Jamison, D., Nugent, H., Gelband, S., Horton, P., Jha, R., Laxminarayan, & Mock, C. Prioridades para el control de enfermedades: Compendio de la 3 a edición. Washington, DC, 2018. https://openknowledge.worldbank.org/handle/10986/29392
14. 14. Organización Panamericana de la Salud. OPS. La carga de los trastornos mentales en la Región de las Américas, 2018. Washington, D.C.: OPS, 2018. https://iris.paho.org/bitstream/handle/10665.2/49578/9789275320280_spa.pdf?sequence=9
15. 15. Leiva-Peña V, Rubí-González P, Vicente-Parada B. Determinantes sociales de la salud mental: políticas públicas desde el modelo biopsicosocial en países latinoamericanos [Social Determinants of Mental Health: Public Policies Based on the Biopsychosocial Model in Latin American CountriesDeterminantes sociais da saúde mental: políticas públicas a partir do modelo biopsicossocial em países latino-americanos]. Rev Panam Salud Publica. 2021 Dec 16;45:e158. Spanish. doi: 10.26633/RPSP.2021.158. PMID: 34934419; PMCID: PMC8678097.
16. 16. Adair CE, McDougall GM, Mitton CR, Joyce AS, Wild TC, Gordon A, Costigan N, Kowalsky L, Pasmeny G, Beckie A. Continuity of care and health outcomes among persons with severe mental illness. Psychiatr Serv. 2005 Sep;56(9):1061-9. doi: 10.1176/appi.ps.56.9.1061. PMID: 16148318.
17. 17. Adair CE, McDougall GM, Beckie A, Joyce A, Mitton C, Wild CT, Gordon A, Costigan N. History and measurement of continuity of care in mental health services and evidence of its role in outcomes. Psychiatr Serv. 2003 Oct;54(10):1351-6. doi: 10.1176/appi.ps.54.10.1351. PMID: 14557520.
18. 18. Hernández, S., Fernández, C. & Baptista, P. Metodología de la investigación. México: McGraw-Hill, 2003.
19. 19. Krach, S. K., McCreery, M. P., & Guerard, J. Cultural-linguistic test adaptations: Guidelines for selection, alteration, use, and review. School Psychology International, 2017; 38(1), 3–21. https://doi.org/10.1177/0143034316684672.
20. 20. Gjersing L, Caplehorn JR, Clausen T. Cross-cultural adaptation of research instruments: language, setting, time and statistical considerations. BMC Med Res Methodol. 2010 Feb 10;10:13. doi: 10.1186/1471-2288-10-13. PMID: 20144247; PMCID: PMC2831007.
21. 21. Escobar-Pérez J and Cuervo- Martínez A. Validez de contenido y juicio de expertos: una aproximación a su utilización. Avances en Medición, 2008; vol. 6, núm. 1, pp. 27-36. Recuperado de http://www.humanas.unal.edu.co/psicometria/files/7113/8574/5708/Articulo3_Juicio_de_expertos_27-36.pdf.
22. 22. Gajardo J, Espinosa T, Muñoz E, Goycolea M, Valdebenito A, Oyarzún S. Traducción y análisis de validez de contenido del instrumento Opening Minds Scale for Healthcare Practitioners (OMS-HC) para la evaluación del estigma hacia la enfermedad mental en profesionales de la salud en Chile. Rev. chil. neuro-psiquiatr. [Internet]. 2021 Jun [citado 2022 Abr 29] ; 59( 2 ): 91-101. Disponible en: http://www.scielo.cl/scielo.php?script=sci_arttext&pid=S0717-92272021000200091&lng=es. http://dx.doi.org/10.4067/s0717-92272021000200091.
23. 23. Tavakol M, Dennick R. Making sense of Cronbach’s alpha. Int J Med Educ. 2011 Jun 27;2:53-55. doi: 10.5116/ijme.4dfb.8dfd. PMID: 28029643; PMCID: PMC4205511.
24. 24. Durbin J, Goering P, Streiner DL, Pink G. Continuity of care: validation of a new self-report measure for individuals using mental health services. J Behav Health Serv Res. 2004 Jul-Sep;31(3):279-96. doi: 10.1007/BF02287291. PMID: 15263867.
25. 25. Organizacio?n Panamericana de la Salud. OPS. DESC – BIREME, 2022. http://decs2020.bvsalud.org/cgi-bin/wxis1660.exe/decsserver/
26. 26. Ware NC, Tugenberg T, Dickey B, McHorney CA. An ethnographic study of the meaning of continuity of care in mental health services. Psychiatr Serv. 1999 Mar;50(3):395-400. doi: 10.1176/ps.50.3.395. PMID: 10096646.
27. 27. Adair CE, Wild TC, Joyce A, et al: Development of a measure of perceived continuity of care for persons with severe mental illness. Presented at the Congress of Epidemiology, Toronto, Ontario, June 13–16, 2001.
28. 28. Ride J, Kasteridis P, Gutacker N, Doran T, Rice N, Gravelle H, Kendrick T, Mason A, Goddard M, Siddiqi N, Gilbody S, Williams R, Aylott L, Dare C, Jacobs R. Impact of family practice continuity of care on unplanned hospital use for people with serious mental illness. Health Serv Res. 2019 Dec;54(6):1316-1325. doi: 10.1111/1475-6773.13211. Epub 2019 Oct 9. PMID: 31598965; PMCID: PMC6863233.
29. 29. Tammes P, Salisbury C. Continuity of primary care matters and should be protected. BMJ. 2017 Feb 1;356:j373. doi: 10.1136/bmj.j373. PMID: 28148480.
30. 30. Hoertel N, Limosin F, Leleu H. Poor longitudinal continuity of care is associated with an increased mortality rate among patients with mental disorders: results from the French National Health Insurance Reimbursement Database. Eur Psychiatry. 2014 Aug;29(6):358-64. doi: 10.1016/j.eurpsy.2013.12.001. Epub 2014 Jan 16. PMID: 24439514.
31. 31. Puntis SR, Rugkåsa J, Burns T. The association between continuity of care and readmission to hospital in patients with severe psychosis. Soc Psychiatry Psychiatr Epidemiol. 2016 Dec;51(12):1633-1643. doi: 10.1007/s00127-016-1287-3. Epub 2016 Oct 25. PMID: 27783129; PMCID: PMC5131080.
32. 32. Reilly S, Planner C, Hann M, Reeves D, Nazareth I, Lester H. The role of primary care in service provision for people with severe mental illness in the United Kingdom. PLoS One. 2012;7(5):e36468. doi: 10.1371/journal.pone.0036468. Epub 2012 May 15. PMID: 22615769; PMCID: PMC3352919
33. 33. Tapsell, A., Martin, K.M., Moxham, L., Bur-ns, S., Perlman, D., & Patterson, C. (2020) Expert by Experience Involvement in Mental Health Research: Developing a Wellbeing Brochure for People with Lived Experiences of Mental Illness. Issues Ment Health Nurs. doi:10.108 0/01612840.2019.1663566.
34.