Development and Psychometric Study of Early Punishment Susceptibility (EPSS)

Abstract

Introduction: Taking into account the concept of Punishment Susceptibility from the Gray’s Reinforcement Sensitivity Theory and human-specific Internal Verbal-Linguistic Activity, the concept of Early Punishment Susceptibility (EPS) is proposed, as there is not instrument to evaluate it. The

Objective: is design and study the adequacy of the items, the internal structure and the reliability of Early Punishment Susceptibility (EPSS). Additionally, some descriptive EPSS data are presented.

Method: First, the items are devised, according to the definition of the concept, to be further valued by experts. During the pilot phase the initial EPSS version was performed with 240 people adults with whom a debugging was made and a first consistency study based on a Factorial First-Pass Analysis. Later, a Confirmatory Factorial Analysis was made with the final version, in a sample with 960 college students.  The adequacy of the items was confirmed and EPSS reliability was studied.

Results: All the items of the final version turned out to be adequate. During the final stage an internal three-factor structure was reported and confirmed (and one in a second order). The factors and the total scale reported adequate internal consistency values.

Conclusions: EPSS, despite most limitations derived from the fact that it was a new construct, it is a proper instrument for evaluating Early Punishment Susceptibility in the college population of Michoacán (Mexico).

Key Words: Punishment, Scale, Psychometric, Reliability and Validity.

Introduction

Susceptibility to punishment and reinforcement are concepts included in the Gray´s Theory of Sensitivity to Reinforcement (TSR) ^(1-2), personality is consistently described with biological findings. Likewise, Gray´s ⁽²⁾ theory considers the existence of three systems supporting neurobiological mechanisms. The systems the theory makes reference to, which modulate learning processes, are as follows: Behavioral Inhibition System (BIS), the Behavioral Activation System (BAS) and the Fight or Flight System (FFS); are known in English as the Acronyms BIS, BAS and FFS ^(3-4). SIC deals with biological sensitivity of response to adverse conditioned stimuli (punishment signs), and also highly new/high intensity stimuli and stimuli evoking innate fear. SIC deals with anxiety traits and modulates learning of inhibition response before adverse stimuli and active avoidance. SAC deals with sensitivity to conditioned appetitive stimulus, thus generating a positive feedback cycle. It is activated when a stimulus associated to reward and punishment stop signs arise. SLH is activated before unconditioned adverse stimuli; thus provoking Fight or Flight responses, which are related with pain and negative affective moods ^(5-6).

However, as TSR is based on animal experimenting and is aimed to always find physiological correlations significantly limited research in humans. In order to study individual differences in SIC/SAC reactivity in humans, the most used self-report instruments are the Sensitivity to Punishment and Reward Questionnaire (SPSRQ) ⁽⁷⁾, which is made up of 48 two-dimensional dichotomic items. The other test is the Behavioral Activation and Behavioral Inhibition Scales (BAI), better known as the BIS/BAS Scale ⁽⁸⁾.  It is made up of 20 items to be responded by means of a Likert-type Scale with four choices. Responses are grouped in two subscales: Behavioral Inhibition System (BIS) comprising seven responses aimed to measure behavioral inhibition tendency before punishment or lack of reward; and Behavioral Activation System (BAS), comprising thirteen items, which, in turn contain three factors measuring pulsions (4 reactives), Search for amusement or pleasure emotions (4 reactives) and responsiveness to reward (5 items). The high positive correlations found between BIS and Neuroticism ^{(9-10-11-12-13)} and Negative Affection are highlighted, as well as those moderate/positive with depression ^(10-11-13) and anxiety ^(9-10-12-13).

After analyzing factors and reactives of both scales regarding punishment, it is surprising that in both instruments only one factor appears generally referring to Punishment Susceptibility or to BIS. On the other hand, many reactives of Carver and White´s BIS Scale ⁽⁸⁾ are biased as these value dysfunctions, such as: “I am too concerned about things” or “I get stuck in my problems” extracted from the Argentinian translation ⁽¹⁴⁾. The same thing happens in the Torrubia Scale ⁽⁷⁾ ¿Do you often take a lot of time to fall asleep thinking about things you have done or things you have to do? or ¿Would it be hard for you to go back to a store in order to complain about, if you realize change returned is not correct? Probably, what these scales most lack of, as they are based on Gray’s Reinforcement Sensitivity Theory ^(1-2), is that they are conceived, basically the least evolved processes/shared with other species are conceived; i.e. Fight or Flight Responses or the Inhibition System are included. Many times these responses refer to immediate situations. When reactives have been made dealing with more complex/evolved processes, such as the foregoing, the presence of dysfunctions is valued. In this research Allman´s proposal is taken into consideration ⁽¹⁵⁾. It states that more evolved systems are settled on older systems, and they are not replaced, but rather overlapped. It is worth mentioning that Internal Verbal-Linguistic Activity related with anticipation skills ^(16-17) rules conduct. For this reason, in this research we start with Gray´s TSR (1070; 1981) only focused on Punishment Susceptibility and taking into account Internal Verbal-Linguistic Activity inherent to human beings in order to propose the concept of Early Punishment Susceptibility (EPS). EPS is defined as the personality trait characterized by the tendency to show behavior influenced by individual punishment susceptibility, regulated by the internal language. Functioning derived from thinking about the possibility of future threats (i.e. anticipated) has been described as highly adaptive, even though it may have flaws and become a pathology ⁽¹⁸⁾. Therefore, it is easier to identify people with higher/lower tendency to be cautious, obedient or fearing to be rejected.

That is why, the objective of this research was create the Early Punishment Susceptibility Scale (EPSS). With this tool we have performed a psychometric study of EPSS, showing the internal structure characteristics, internal consistency and descriptive data of the scale.

The Project for this research was evaluated and approved by the Ethics Committee of the Universidad Michoacana de San Nicolás de Hidalgo. The research was carried out in three phases: The first one was aimed to devise and ranking of items. The second phase (pilot phase), debugged the instrument and the internal structure was studied in an exploratory manner. The third phase (final phase), applied the final version of the instrument, and adequacy of the items and internal structure of the scale was confirmed.

PHASE 1: IMPLEMENTATION AND VALIDATION OF EPSS CONTENT

The items based on EPS definition were developed and the following elements proposed by the authors of this work were taken into account: activation easiness before a potential future threat, avoidance tendency, discomfort before the possibility of adverse stimulus, tendency to precaution/prevention, welfare before lack of an adverse stimulus and tendency to be obedient or trustworthy.

For analyzing content validity three judges (psychometric experts) were asked to assess writing and correction of each one of the statements, considering “EPS” construct and the fact this instrument is aimed to the general population general. A total of 35 were approved by the judges, which became part of the first EPSS version.

PHASE 2: PILOT STUDY

Method

Participants
240 people participated in the pilot phase (178. 74.2% were women) with age ranging from 17 to 64 years old (M=25.38; OF = 8.63).

Instrument
The Scale has 35 EPSS items (initial version). Each has 5 Likert-type answer choices, dealing with frequency (never, almost never, sometimes, almost always, and always) scores ranged from 1 to 5.

Procedure
During the Pilot phase, EPSS of 35 reactives for 240 adult people was implemented. Passersby were invited to voluntarily participate in this research. They were requested to provide their information about sex, age, marital status and had to sign an informed consent. The average duration of the test was of 10 minutes. Data retrieved were analyzed with SPSS Program version 2.0. A Factorial First-Pass Analysis was made by using the Maximum Likelihood Procedure Method with Oblimin rotation. Likewise, the KMO Index and the Barlett´s Sphericity Test were used. Internal consistency was studied, by means of the Cronbach Alpha Index; adequacy of the reactives was studied by means of the corrected correlation with the Total Scale (without the item itself) and the analysis of Cronbach´s Alpha Value, if the item is eliminated.

Results
In the Factorial First-Pass Analysis, Bartlett´s Sphericity Test was reported to be significant (χ2= 839.99 gl=78; p< .001), as well as the KMO test =0.788, which suggests applicability of the factorial analysis on data. 22 reactives were eliminated due to the following criteria; 1) not to clearly weight some factors, 2) not reporting a minimum correlation of .30 with the total of the scale or 3) to increase the value of Cronbach´s Alpha from the total of the scale or from the factor where it could be located. Finally, 13 items reported adequate psychometric properties which made up the final version. On the other hand, 3 factors were reported to obtain self-values (Eigen values) with values higher than the Unit that explained 55.89% of the variance. The three factors reported theoretical coherence and were clearly taggable; precaution/prevention (items 3,5,7,9 and 12) explain 28.17% of the variance, fear to rejection (1,8,10 and 13) explaining 17.71% and obedience/compliance (2,4,6 and 11) explaining 10.00%.

Regarding internal consistency, the Cronbach´s Alpha value of the total scale was .728 and that of factors were precaution/prevention (α=.790), fear to rejection (α=.739) and obedience/compliance (α=.694).

PHASE 3: FINAL VERSION STUDY

Method

Participants
In the final phase the sample included 960 students from various college careers from the Universidad Michoacana de San Nicolás de Hidalgo, with ages ranging between 18 to 56 years old (M=21.31; DE=3.77) from which 566 (59.0%) were women who lived in Morelia city (México). Most of them reported to be single (903; 94.1%).

Instrument
EPSS (final version) includes 13 items, with 5 Likert-type response choices, whose theoretical minimum and maximum scores are 13 and 65, respectively. The scale has three factors: compliance/obedience (α=.694), fear to rejection (α=.739) and precaution/prevention (α=.790). Internal consistency of the total scale is .728, See Annex.

Procedure
EPSS of 13 items was applied to 960 college students from the UMSNH. Permission was requested to the Head of the various faculties. Later the items were applied in groups, anonymously and voluntarily. The scale XXXX, along with sociodemographic data (sex, age and marital status) with an approximate duration of 7 minutes. All participants signed an informed consent.

The program SPSS version 2.0 ⁽¹⁹⁾ was used to perform the analysis. For the Confirmatory Factorial Analysis (CFA), the program EQS 6.1. ⁽²⁰⁾ was used. Descriptive median, standard deviation, asymmetry and kurtosis were used, as well as other indexes, such as the Pearson´s correlation and Cronbach´s Alpha. CFA was made by means of Structural Equations Analysis and it was made by using the Robust Maximum Likelihood Method (Robust ML). the following indicators were used: Satorra-Bentler chi-square statistic (χ2), its significance, the chi-square divided by the degrees of freedom, non-normed fit index (NNFI), normed fit index (NFI) Comparative Fit Index (CFI), Root Mean Squared Error of Approximation (RMSEA). The model considers acceptance, according to the following criteria: χ2 value does not turn out to be significant; the χ2 index, divided by degrees of freedom; values ≤ 5.0 indicate a good adjustment of the model; NNFI, NFI and CFI > .90, indicate adequate adjustment, and > .95 excellent; RMSEA < .08, even though it is an excellent adjustment of the model, it is reached when RMSEA < .05 ^(21-22).

Results
The reactives median ranged among 2.51 (reactive 21) and 3.91 (reactive 11) when the theoretical value ranges between 1 and 5. The standard deviation of the items were distributed between .81 and 1.22. The asymmetry values and kurtosis of the items were found between 1 and -1, which suggests that all of them have a normal distribution. The correlation of each reactive with the corrected total score, not considering the reactive itself, were all above .30 except item 1. Additionally, only item 1, when it was eliminated, it slightly increased Cronbach´s Alpha value of the total scale (Table 1). Item 1 was kept, because it properly correlates with the total of its factor (fear to rejection) and significantly contributes in the Alpha Value of such factor. In fact, all the items were correlated with the factor score (except the item itself) with values higher than .4. In the factor precaution/prevention correlations ranged between .566 and .637; in factor fear to rejection it ranged between .504 and .634; and in the factor obedience/compliance it ranged between .439 and .678. All the items, when eliminated, made that Cronbach´s Alpha of the factor decreased.

Table 1: Description of the item “Early Punishment Susceptibility Scale”.

Regarding adequacy adjustment three models were tested by using CFA. The three-factor analysis and one in a second order; three related/unifactorial factors. Regarding χ2, the three models obtained significant values. However, the value of χ2/gl, the three-factor model with one of a second order was the only one resulting in < 5.0; although the three-related-factor model got a close score. On the other hand, the other indexes turned out to be adequate in the models including three factors reporting an acceptable adjustment. But the model including 3 factors and one of a second order was reported to obtain better adjustment adequacy than the other two, See Table 2 and Figure #1.
 

Table 2: Adjustment Indexes of the models based on the Confirmatory Factorial Analysis.

Nota. Indexes; χ2 = chi-square; gl = degrees of freedom; χ2 /gl; chi-square divided by the degrees of freedom; NNFI=non-normed fit index (NNFI); NFI= normed fit index (NFI); Comparative Fit Index (CFI); RMSEA = RMSEA= approximation error of the quadratic mean; IC = Confidence interval.

Figure #1: Three-factor Model, and one of a second grade of the Early Punishment Susceptibility Scale (EPSS).

High correlations are reported among the three factors and the total scale. The correlation between the factor precaution/prevention and obedience/compliance was moderate. However, correlations between the factor fear to rejection and the other two factors, turned out to be low (See Table 3).

Table 3: Pearson´s Correlations between the full EPSS Scale and its Factors

Nota. *p < .01, **p < .001

In order to evaluate the internal consistency, Cronbach´s Alpha was used. Three factors were reported to have adequate values: precaution/prevention (α=.812), fear to rejection (α=.766) and obedience/compliance (α=.757). Internal consistency of the total scale turned out to be adequate as well (α=.791).

Median was 45.26; DE was = 6.89; mode was 48; the minimum value was 18; and the maximum value was 64. See frequencies and percentages in Table 3. Asymmetry value was     -.479; kurtosis was .711, which suggest a normal distribution of the total EPSS score. Table 4 displays percentiles of the total score and EPSS factors.

Table 4: Description of the EPSS Percentiles and their factors.

Discussion
This work was aimed to elaborate and analyze the internal structure/internal consistency of a Scale aimed to evaluate Early Punishment Susceptibility. First, we must state that all items were approved by three judges who are experts in psychometrics, and who were told about the definition and were given an explanation of the EPS Construct, which provides content validity evidence to the scale.

The internal structure arising from AFE was made up of three factors; precaution/prevention, obedience/compliance and fear to rejection, which was further confirmed, based on the CFA. The existing correlations of the three factors, with the total of the scale are high; these are moderate, between the factors precaution/prevention and obedience/compliance. However, the low correlation between the factor fear to rejection and the other two factors may become a surprise, but it is worth mentioning the factor refers to an emotion; and the factors precaution/prevention and obedience/compliance refer to conduct styles. Additionally, the factor fear to rejection is theoretically more directly related with avoidance conduct (for instance: 22-23), it is a factor that was included in the items generation, but such items were not kept. We must comment that in the future we could include other potential factors, such as avoidance, cognitive hypervigilance, among others as this is a new construct and this study represents the first approximation.

Internal consistency of the total scale and of the three factors turned out to be adequate, especially if we take into account the low number of items of the total scale and that of the factors.

A significant limitation in this study is the lack of evidence for criterion validity. In the future it would be interesting to study if high/low scores are correlated with various predicaments or disturbances. Likewise, it is necessary to study EPSS correlation with other variables, both in terms of personality and also psychopathology presence in the line of the research made by several authors regarding Punishment Susceptibility or BIS ^{(9-10-11-12-13)}. For instance, high scores are probably correlated with high levels of anxiety and pathological concerns; and low scores could be correlated with procrastination and problems with the authority, among others.

Other limitations of this research deal with the sampling, which was incidental and only with college students’ population (in the final phase). For future research it would be desirable to carry out a random sampling, as much stratified as possible, according to social-economical levels. Likewise, representativity is influenced by the fact that the sample was extracted from the population of Michoacán, and that is only one State in Mexico. It is necessary to study EPSS properties in other states of the country.

On the other hand, if we take into account that the EPS construct is described as a personality trait, we must study test-retest reliability, expecting that the scores of one person are very similar when evaluated one or two months later.
Despite the foregoing limitations, we may conclude that the Early Punishment Susceptibility (EPSS) Scale obtained adequate indicators, regarding its construct validity and internal consistency.  EPSS is useful as an instrument aimed to evaluate the personality trait named as Early Punishment Susceptibility (with its factors) which may be quite interesting, both in the research area and also in the application.

Acknowledgements
This research was funded by the Consejo Nacional de Ciencia & Tecnología (CONACYT).

References

1. Gray JA. The psychophysiological basis of introversion-extraversion. Behav Res Ther 1970;8:249-266.
2. Gray JA. (1981). A critique of Eysenck’s theory of personality. En H.J. Eysenck, editor. A model for personality: New York; Springer. 1981. p. 246-276.
3. Gray JA, McNaughton N. The Neuropsychology of Anxiety: An Enquiry into the Functions of Septo-hippocampal System (2nd Ed.). Oxford: Oxford University Press. 2000.
4. Pérez F, Blasco CM, Pérez RL, Bernabeu F, Esteve Z. Sensibilidad al refuerzo y al castigo e impulsividad en criminales muy violentos. Una evaluación psicopatológica cualitativa a partir de varios sujetos condenados. Behav Law J 2015; 1:49-62.
5. Corr PJ. Reinforcement sensitivity theory (RST): introduction. En P. J. Corr Editor., The reinforcement sensitivity theory of personality, (pp. 1-43). Cambridge: Cambridge University Press; 2008. p. 1-43.
6. Pascual D, Pascual T, Redondo M, Pérez Nieto MA. Sensibilidad a la recompensa y al castigo, personalidad, impulsividad y aprendizaje: un estudio en un contexto de violencia íntima de pareja. Clínica y Salud 2014; 25: 167-174.
7. Torrubia R, Avila C, Moltó J, Caseras X. The Sensitivity to Punishment and Sensitivity to Reward Questionnaire (SPSRQ) as a measure of Gray's anxiety and impulsivity dimensions. Pers Individ Dif 2001;31: 837-862.
8. Carver CS, White TL. Behavioral inhibition, behavioral activation, and affective responses to impending reward and punishment: the BIS/BAS scales. J Pers Soc Psychol 1994; 67; 319-333.
9. Caseras X, Avila C, Torrubia R. The measurement of individual differences in behavioural inhibition and behavioural activation systems: a comparison of personality scales. Pers Individ Dif 2003; 34: 999-1013.
10. Campbell-Sills L, Liverant GI, Brown T. Psychometric evaluation of the behavioral inhibition/behavioral activation scales in a large sample of outpatients with anxiety and mood disorders. Psychol Assess 2004;16:244-254.
11. Jorm AF, Christensen H, Henderson AS, Jacomb PA, Korten AE, Rodgers B. Using the BIS/BAS scales to measure behavioural inhibition and behavioural activation: Factor structure, validity and norms in a large community sample. Pers Individ Dif 1999; 26: 49-58.
12. Müller JM, Wytykowska AM. Psychometric properties and validation of a Polish adaptation of Carver and White’s BIS/BAS scales. Pers Individ Dif 2005; 39: 795-805.
13. Segarra P, Poy R, López R, Moltó J. Characterizing Carver and White’s BIS/BAS subscales using the Five Factor Model of personality. Pers Individ Dif 2014; 61:18-23.
14. Martinez MV, Zalazar-Jaime MF, Pilatti A, Cupani M. Adaptación del Cuestionario de Personalidad BIS BAS IPIP a una muestra de estudiantes universitarios argentinos y análisis de su relación con patrones de consumo de alcohol. Avances en Psicología Latinoamericana 2012; 30:304-316.
15. Allman JM. Evolving brains. New York: Scientific American Library, 1999.
16. Deacon TW. The symbolic species. New York: Norton company, 1997.
17. Maynard-Smith J, Szathmar E. The major transitions in evolution. Oxford: oxford university Press, 1995.
18. Padrós Blázquez F, López Solà C, Cruz García MA, Martínez Medina MP. PAMTA worry model (perception of future threat, activation, motivation, thought & action): Functional and pathological worry. Rev Argent  Clín Psicol 2019; 28:190-198.
19. IBM Corp Released, IBM SPSS statistics for Windows, version 22.0. Armonk, NY: IBM Corp, 2013.
20. Muthén LK, Muthén BO. Mplus user’s guide (Version 7). Los Angeles; Muthén & Muthén, 2011.
21. Bentler PM. Comparative fit indexes in structural models. Psychol Bull 1990;107; 238-246.
22. Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Struct Equ Model 1999; 6:1-55.
23. Boselie JLM, Vlaeyen JW. Broadening the fear-avoidance model of chronic pain?. Scand J Pain 2017;17:176-177.
24. Peeters G, Bennett M, Donoghue OA, Kennelly S, Kenny RA. Understanding the aetiology of fear of falling from the perspective of a fear-avoidance model-A narrative review. Clin Psychol Rev 2020; 101862

Annex

DIRECTIONS: Please read carefully all statements to be next presented. Answer the first thought that comes to your mind, marking with an X the choice you feel identified with; whether it is Never, Almost-never, Sometimes, Almost-always or Always.

Items 3,5,7,9 and 12 make up the factor “precaution/prevention” (F1); Items 1,8,10 and 13 make up the factor “fear to rejection” (F2); finally, items 2,4,6 and 11 make up the factor “obedience/compliance” (F3).