text:
Introduction Juvenile violent offenders are among the most dangerous offenders in the criminal justice system
Juvenile violent offenders are among the most dangerous offenders in the criminal justice system, responsible and convicted for a variety of severe criminal offenses such as robbery, assault, rape, and homicide. Thus, the assessment of juvenile offenders for risk of recidivism, treatment planning and treatment effectiveness has invoked considerable research interest. Only recently has research into the development and validation of juvenile assessment instruments, implementation of different treatment approaches and intervention programs as well as studies of effectiveness of treatment with juvenile offenders advanced considerably.
The risk-needs-responsivity (RNR) model is considered the most influential model for determining offender risk-assessment and treatment. Over the last few decades the model of effective correctional intervention has become the most commonly used paradigm of corrections and juvenile justice (Andrews, Bonta, & Wormith, 2011; Howell, Lipsey, & Wilson, 2014; McGrath & Thompson, 2012; Peterson-Badali, Skilling, & Haqanee, 2015; Vose, Lowenkamp, Smith, & Cullen, 2009). As suggested by its name, the model proposes the following three principles at the essence of effective programming:
the risk principle addresses the question of who should be treated and states that the level of service provided to the offender should match the offender's risk to reoffend (high-risk offenders vs. low-risk offenders). Thus, high-risk offenders should receive intensive corrections, while interventions for the low-risk offenders should be minimized, since interventions with low-risk offenders can induce paradoxical effects such as increment in recidivism (Gendreau, Smith, ; French, 2006; Ogloff ; Davis, 2004, Andrews ; Dowden, 2006);
the need principle addresses what should be treated and states that treatment should identify and target criminogenic needs (also called dynamic factors) – e.g., criminal thinking patterns
the responsivity principle addresses how should be treated and highlights the importance of applying correctional practices effective to change criminal behavior (e.g., cognitive-behavioral and social learning approaches). Interventions should be highly individualized to match treatment to client characteristics. In particular, treatment should be adapted to the offenders' strengths, abilities, motivation, personality, and bio-demographic characteristics such as gender, ethnicity, and age (Andrews & Bonta, 2010).
The three principles of risk, need, and responsivity are the core, and together have come to be known as the RNR (risk–need–responsivity) model of offender rehabilitation. According to the RNR model, offending is an outcome of a combination of static and dynamic risk factor, both predictive of reoffending. While static, historical risk factors (such as age at first offence, prior criminal history) are features not amenable to deliberate intervention, dynamic risk factors (e.g., procriminal attitudes such as thoughts, values and beliefs supportive of criminal behavior) can change in both directions. For example, one can develop prosocial or procriminal attitudes. The following dynamic factors have been identified as significant to reducing offending: substance use, antisocial cognition, antisocial associates, family and marital relations, employment, and leisure and recreational activities (Andrews & Bonta, 2006). Given that dynamic risk factors are considered responsible for increased risk, they have also been called criminogenic needs. Although static risk factors do predict long-term recidivism (e.g., Harris & Rice, 2003), the assessment of change in offender risk level, however, requires the consideration of dynamic (changeable) risk factors. Since they are sensitive to changes and responsive to interventions, focusing the treatment on criminogenic needs (dynamic factors) is considered a fundamental component of the RNR model (Lowenkamp & Bechtel, 2007; Simourd, 2004). Evidence shows that recidivism can be reduced up to 10% when criminogenic risk factors are addressed in a community setting (Andrews & Bonta, 2010; Lipsey, 1995; Lösel, 1995). Moreover, intervention programs addressing dynamic factors and explicitly adopting the risk, needs, and responsivity principles, show possible reduction in recidivism up to 50% (Andrews & Bonta, 2010; Andrews, Bonta, & Hoge, 1990). In a meta-analysis, including 400 treatment studies of juvenile delinquents and yielding 443 effect size estimates, Lipsey (1989) evidenced that treatment, on average, had a 10% reduction on recidivism. After controlling for methodological (e.g., sample size, attrition) and treatment variables (e.g., duration, evaluator involvement), reduction in recidivism increased by 30%. Since Lipsey's (1989) meta-analysis, the overall effectiveness of offender treatment was confirmed by 40 more meta-analyses.
The RNR model appears to generalize across a variety of settings, criminal behavior, and offender subtypes. Adherence to the RNR principles is associated with reduced re-offending in both community and custodial settings. Meta-analytic studies have found the principles to apply to youth (Andrews, Zinger et al., 1990; Dowden ; Andrews, 1999a), women offenders (Dowden ; Andrews, 1999b), and minorities (Andrews ; Bonta, 2006; Andrews, Dowden, ; Rettinger, 2001). Adherence to the principles in treatment programs also appear relevant to violent offending (Dowden ; Andrews, 2000), prison misconducts (French ; Gendreau, 2006), gangs (Di Placido, Simon, Witte, Gu, ; Wong, 2006), and even sexual offending (Hanson, Bourgon, Helmus, ; Hodgson, 2009).
HAB as a criminogenic need/dynamic factor: definition, relation to aggressive behavior and assessment approaches
The phenomenon of juvenile delinquency and violent behavior highlights the need of exploring the components of juvenile violation on a behavioral and cognitive level. Committing a violent crime addresses not only the potential for aggressive behavior, but equally importantly personal attitudes and values supporting aggression and crime such as high tolerance for deviance, rejection of the validity of low, interpretation of wide range of stimuli as hostile, resentment and/or defiance etc. Discovering the factors associated with the manifestation and maintenance of aggressive behavior is an important step in understanding youth crime. Therefore, one of the well-recognized dynamic factors/criminogenic needs that reinforce participation in criminal activity is the subdomain antisocial attitudes or antisocial cognitions (Walters, 1990). Antisocial thought patterns, such as justification, rationalization, entitlement and hostility have shown to be associated with criminal history (Healy ; O'Donnell, 2006; Palmer & Hollin, 2004). Walters (1995, 2003) and Henning and Frueh (1996) offer that cognitive thinking errors reinforce criminal lifestyles through self-interest, minimization of prosocial activities, denial of responsibility for behavior, and pleasurable or deviant thoughts about criminal activity. The current study focuses of the assessment of one particular key component of the antisocial attitudes, namely the hostility-prone cognitions.
Hostility-prone cognitions are shown to be a very common symptom among young offenders. They represent the individual's tendency to over-attribute hostile intent to the behavior of others, even in situations where hostile attributions are unjustified, for example, when the actual intentions of others are benign in their nature or when the social situation is ambiguous (Dodge, 1980; Dodge ; Frame, 1982). Furthermore, hostility-prone attributions are shown to affect the encoding of social cues and, thus, to lead to impulsive or aggressive overreactions that are incongruent with the situation (Crick ; Dodge, 1996; Dodge ; Coie, 1987).
An extensive number of studies suggest that individuals with aggressive tendencies often feel provoked by ambiguous behavior and attribute a negative, hostile intent to other people's actions, particularly in uncertain, ambiguous circumstances (Crick & Dodge, 1994; Dodge & Frame, 1982; Dodge & Newman, 1981; Dodge, Price, Newman, & Bachorowski, 1990; Van Oostrum & Horvath, 1997; for overview and review, see Dodge, 2006; Orobio de Castro, Veerman, Koops, Bosch, & Monshouwer, 2002). This phenomenon is referred to as hostile cognitive/attribution bias (HAB). HAB relates mainly to the distorted perception of other's intentions in an interpersonal social context. Hostility-prone attributions of intent and their relation to violent behavior, aggression and crime appears to generalize across a variety of settings and samples. Studies have found HAB to apply to children (e.g., Dodge ; Coie, 1987), adults, as well as in forensic samples – both juvenile (Dodge et al., 1990; Slaby ; Guerra, 1988) and adult offenders (Seager, 2005; Serin, 1991; Vitale, Newman, Serin, ; Bolt, 2005).
HAB shows a close semantic relationship with paranoia and suspicion and shares notable similarities with paranoid patterns of thought. Some even claim that HAB is one of the varied cognitive processes – along with an attention bias for threat and jumping-to-conclusions-bias – that are likely components of a paranoid thinking style in both normative and atypical developing individuals (Tone ; Davis, 2012). A few recent studies showed evidence that hostile intent attribution biases constitute a core element of paranoid thought, both in healthy samples (Combs, Penn, Wichert, ; Waldheter, 2007; Fornells-Ambrojo ; Garety, 2009) and in adults with persecutory delusions (Combs et al., 2009). Although paranoia primarily indicates a clinical psychopathological condition, the term now signifies a tendency toward excessive or irrational suspiciousness and distrustfulness of others. HAB, in particular, is defined to represent the non-pathological, normative aspects of the paranoid thinking style/interpersonal suspiciousness and, thus, is widely used in ordinary, nonclinical discourse.
There are different methods of assessing hostility-prone attributions. Standardized, clinic-based measures of hostility as a personality feature and irrational suspiciousness have been incorporated routinely into the diagnostic of HAB. Purportedly, they provide objective data less influenced by other factors (e.g., well differentiated language skills needed in other measures such as semi-projective methods) that might otherwise bias participants' report. On the other hand, they are highly affected by the self-perception of the participants and often contain socially desirable responses.
The most popular procedure of assessing HAB is the vignettes method proposed by Dodge (1980). Based on his idea of testing HAB using social scenarios, several different versions of this method have been developed for the purposes of the experimental research. Basically, the vignettes represent social situations, in which the subjects experience a negative outcome (for instance, being excluded from a group game). Since there is no additional information about the circumstances or intentions of others involved in the social interaction, the situation becomes highly ambiguous. The subjects are asked to decide whether the other person has acted on purpose or by accident and, thereby, to indirectly attribute a hostile or benign intent to the other individual's actions. The vignettes dominate the later studies and have already established themselves as a reliable instrument, which makes it possible to distinguish between aggressive and nonaggressive children (e.g. Dodge ; Somberg, 1987). In forensic samples, vignettes are the most commonly used instrument for assessing hostility-prone cognitions and are usually preferred to other experimental methods such as self-assessment instruments or a real-life social experiment.
However, in a realistic social context the individuals are often confronted with numerous highly ambiguous factors when judging someone's behavior as hostile or benign. For humans the endowment to observe and control the environment efficiently for signs of threat to oneself is critical for their survival (Ledoux, 1996). In many clinical disorders, however, individuals respond inadequately to threat cues, developing a pattern of hypervigilance or attentional bias toward threat (Gotlib et al., 2004; Mogg & Bradley, 1998; Werthmann et al., 2011) that in turn may negatively bias interpretation of ambiguous cues (White, Suway, Pine, Bar-Haim, Fox, 2011). Attention biases for threat have been repeatedly evidenced in the context of anxiety (for a review, see Bar-Haim, Lamy, Pergamin, Bakersman-Kranenburg, & van IJzendoorn, 2007). Some studies suggest further that adults vulnerable to paranoid thought, both those with clinically significant persecutory delusions (e.g., Bentall & Kaney, 1989; Kinderman, 1994; Leafhead, Young, & Szulecka, 1996) and those who endorse normal-range paranoid ideation (e.g., Arguedas, Green, Langdon, & Coltheart, 2006), show a similar bias to over attend to different types of threat cues, even in the absence of real danger.
Experimental research suggested that the impaired ability to decode important social signals might be a probable reason for individuals to interpret another's behavior inaccurately and attribute hostile intent (Schöneberg ; Jusyte, 2014). Studies investigating the link between HAB and the processing of facial expression found that subjects with high scores in hostility perceived happy and neutral faces as less friendly (Knyazev, Bocharov, ; Slobodskoj-Plusnin, 2009) and that a generalized attributional bias to infer hostility from various classes of unambiguous social stimuli becomes more pronounced as aggressiveness increases (Nasby, Hayden, ; De Paulo, 1980). Schönenberg ; Jusyte (2014) found that aggression is associated with a strong preference to interpret ambiguous stimuli containing proportions of an angry expression as hostile, while there was no evidence for a generally biased interpretation of distress cues under conditions of uncertainty. Aggressive individuals, as compared to controls, not only (mis)interpreted ambiguous facial cues as hostile, but also showed a strong tendency to systematically overrate the perceived intensity of anger (Schönenberg ; Jusyte, 2014).
Multimethod/mixed method assessment
Adolescent offenders with chronic violent aggressive behavior demonstrate impairments on several key dimensions of cognition, especially social perception, interpretation of ambiguous social situations, attitudes and beliefs, problem solving strategies, etc.
Currently, there is no standard, easily administered test battery that specifically and efficiently assesses the important cognitive deficits in juvenile violent offenders. Instead, current assessments differ widely in content, duration, procedures, and implementation. The limited generalizability of the findings across studies has reduced the ability of clinicians and researchers to make clear conclusions about the applicability of the assessment tools as well as their capacity to reflect dynamic changes during treatment or intervention. Another disadvantage of current assessments of HAB in juvenile violent offenders is that the majority of the studies apply one and the same assessment technique, the vignettes, as well as one and the same research approach, the quantitative paradigm. A complex phenomenon such as HAB requires a multimethod assessment approach in order to insure a more precise understanding for the construct and more comprehensive and exhaustive assessment of its facets.
These limitations suggest the need for a measure specifically designed for use in studies with forensic samples. The availability of a quick and efficient tool for measuring HAB in juvenile violent offenders could be an extremely useful guide for clinicians making decisions about potential interventions and for researchers implementing clinical trials to assess treatment effects. At present, the German-speaking countries lack established assessment techniques able to detect hostile cognitive distortions. However, there are some implicit, semi-projective instruments as well as objective personality inventories, which may be useful for the assessment of hostile cognitions and contribute to diagnostics and therapy planning. However, due to the lack of standardized methods for the German population, it is necessary to create or adapt some of the tools for the purpose of this study.
With that focus, we aimed to study HAB applying different assessment and analyze approaches, testing the ability of structured objective questionnaires, semi-projective vignettes as well as a computer-based experimental perception task to assess HAB as well as the sensitivity of these techniques to measure changes occurring during treatment. In addition, this study recognizes the need and importance of procedures specifically developed for forensic sampling, and thus seeks to develop a practical and easy to administer multimethod test battery and to discuss its implementation for the scientific research and forensic practice.
Why mixed method?
The evidence gained by exploring HAB multimethodically could be used as a basis for developing more relevant and specific treatments and rehabilitation strategies for the affected individuals as well as for testing intervention effects. Since HAB proved being a complex phenomenon, consisting of perceptual, situational and individual aspects, it seems important to develop an exhaustive assessment battery adequate to measure the diverse elements that HAB incorporates. In addition to external assessment procedures, objective data and file analyzes, we claim that self-assessment methods are also required in order to include the important dimension of the self-experience of the subject in the diagnostic process. Only a fair mixture of verbal (e.g., questionnaire) and non-verbal (eg, images) as well as explicit and implicit procedures can insure a comprehensive survey of cognitive hostile attitudes that matches research standards. Hence, such battery should consist of qualitative, quantitative and experimental methods.
Hence, the present dissertational study integrated mixed quantitative and qualitative research, acknowledging that combined approaches are best suitable for assessing complex phenomena in social science research since they can provide real-life contextual understandings and multi-level perspectives on diverse research questions (e.g., see Johnson, Onwuegbuzie, ; Turner, 2007). The main advantage of qualitative research is its understanding of underlying reasons, beliefs, and motivations. It provides insights in people's individual realities and uncovers inclinations in thinking and experiencing. The statistically analyzed quantitative research is used to quantify behaviors, attitudes, beliefs, and other defined variables, to test hypotheses, to examine relationships among variables, to establish (probable) causality and to generalize results from a sample to a larger population. Combining both types of data in our research, we sought to view problems from multiple individual micro-perspectives to enhance and enrich the meaning of a singular generalized macro-perspective. Merging quantitative and qualitative data allowed us to develop a more complete understanding of HAB; to compare, validate, and triangulate results; to provide illustrations of context for trends; and to examine processes/experiences along with outcomes (Plano Clark, 2010). In addition, mixed method allows us to create a good test battery building the evidence found through different analyze approaches. We started with qualitative analyze, followed by a quantitative phase. That stepwise research design allowed us to develop a test battery, informed by qualitative findings but generalized for the sample needs.
Vignettes as a study approach have now become the most common method and a standard procedure to assess hostile cognitions. Essentially, this method consists of a series of stories (often referred to as vignettes or social scenarios), representing a problematic frustrating situation, in which, due to the actions of others, the subjects are exposed to a negative outcome. The situations are characterized by ambiguity of the intentions of the provocateurs which allows the subjects to produce diverse interpretations of the situations revealing their attitudes and beliefs regarding the intentions of others.
Contrary to the mainstream application of the vignettes as a quantitative assessment tool, in study 1 we adapted a qualitative assessment approach and explored how individuals perceive, construct, and interpret ambiguous social situations and how they attribute intent to an ambiguous stimulus. Instead of categorizing participants' responses quantitatively as "hostile" or "nonhostile.", we focused on investigating respondent's subjective understanding of diverse social situations and the attributes they used to describe a particular behavior. By examining the specific elements in participant's responses qualitatively with regard to social perception, we sought to develop a better understanding of how people perceive ambiguous situations, if they relate interpretations of behavior of others as directed toward the self or personality characteristics of the other, and given these interpretations, how they might react. Furthermore, where hostility was detected, we aimed to identify its key components and the ways it was constructed in participants' responses.
This could be also essential for individualizing treatment methods and adjusting interventions to the individual's personal needs. With these aims, we conducted a qualitative thematic analysis.
In study 2, we attempted to test the dynamic factor HAB in a real-life experimental cognitive task. Based on a social cognition paradigm, we created a computer-based experimental task representing a social human interaction with ambiguous stimuli (facial expression and gaze direction). This experimental approach is evidence-driven and arises from diverse findings showing that the impaired ability to decode important social signals such as facial expressions might be a probable reason for individuals to interpret another's behavior inaccurately and to attribute hostile intention (Burt, Mikolajewski, & Larson, 2009; Nasby et al., 1980; Schönenberg and Jusyte, 2014).
Although a biased face perception could modulate an individual's reaction to ambiguous stimuli, there is one more factor, which could be essential for the display of socially inappropriate or aggressive behavior. This is the tendency to perceive and interpret social stimuli as being directed to oneself. An example of an interpersonal factor, indicating whether someone refers to us or not, is the gaze direction. Thus, the abnormal processing of gaze cues in social interactions may induce irritation, annoyance, or feelings of being provoked and, thereby, trigger an aggressive response. Therefore, the tendency to misinterpret gaze directions may at least partly underlie aggressive–impulsive behavior in susceptible individuals.
To our knowledge, there are no studies investigating this phenomenon in a population of violent offenders. As a result, the present study sought to investigate whether highly hostile violent juvenile offenders do feel more looked at. First, we addressed the question of whether incarcerated juvenile violent offenders show a self-referential bias for preferable processing of particular faces and whether emotional expressions influence the general interpretation of gaze direction. Considering the findings in the sphere of social cognition and emotion processing in violent offenders, we hypothesized a positive relation between a self-directed perception of gaze directions and hostility, especially in the presence of negative facial expressions.
The great overall dissertation project
For assessment tools to adequately measure transformations, reflect real changes in the psychological construct and evaluate treatment effects it is necessary for the intervention program to be proven effective in reducing the target symptomatology. When sure that the applied intervention works, then it is possible for the evaluative tool to be tested for its accuracy to capture changes in the latent variable.
In our studies, all participants were recruited from The Social Therapy Department (STD) of the Juvenile correction center. STD provides offenders with a multimodal cognitive-behavioral treatment based on the RNR model and, in particular, on the determined dynamic risk factors (Andrews ; Bonta, 2010; Gendreau, 1996). The program is directed toward changing distorted or dysfunctional cognitions (cognitive restructuring) or teaching new cognitive skills and involved therapeutic techniques typically associated with CBT, i.e., structured learning experiences designed to affect such cognitive processes as interpreting social cues, monitoring one's own thought processes, identifying and compensating for distortions and errors in thinking, reasoning about right and wrong behavior, generating alternative solutions, and making decisions about appropriate behavior. In addition to the individual treatment plan, of central importance is the mandatory participation in the program Reasoning & Rehabilitation by Ross, Fabiano and Ross (1986). This is a cognitive-behavioral group program developed for acquisition and training of cognitive and social skills. The interventions aim to provide the offenders with alternative cognitive restructuring skills, interpersonal problem solving and social skills, emotional control, moral reasoning skills and victim empathy, behavioral modification and relapse prevention and as well as to ensure a successful reintegration into a stable social environment including suitable housing and employment opportunities.
Various meta-analyses have found cognitive-behavioral therapy (CBT) as an effective intervention for reducing the risk of subsequent reoffending and criminal behavior of juvenile and adult offenders with recidivism reduction rates between 20-30% (e.g. Pearson et al., 2002; Wilson et al., 2005; Lipsey and Landenberger, 2005) up to 55% (Lipsey et al., 2001). Furthermore, the various types or brand names of CBT such as Reasoning and Rehabilitation (Ross and Fabiano 1985), Moral Reconation Therapy (Little and Robinson 1986), Aggression Replacement Training (Goldstein and Glick 1987), the Thinking for a Change curriculum (Bush et al. 1997), and the Cognitive Interventions Program (NIC, 1996), showed no significant differences in the effectiveness of the treatment (Landenberger and Lipsey, 2005 – article I have on the computer). It thus appears to be the general CBT approach, and not any specific version, that is responsible for the overall positive effects on recidivism. It suggests that any representative CBT program that is well-implemented might have results in practice that approach the very positive effects on recidivism produced by the most effective programs documented in the available research studies. Moreover, CBT was found to be age insensitive and proved as effective for juveniles as adults and thus should be useful in both juvenile justice and criminal justice settings (Landenberger and Lipsey, 2005).
The greater dissertation project was devoted to exploring changes in criminogenic needs in the areas of antisocial cognition, concretely the concept of hostile attributions. Using a proven effective correctional intervention program (cognitive-behavioral treatment, R&R-training), the research used a multimethod data collected at two-time points in 12 months to assess if HAB is amenable to change at both aggregate and individual level.
In evaluation studies, data-collection methods are widely used to measure improvement or deterioration in a psychological construct during or after treatment. Usually, evaluation studies of treatment effectiveness use repeated measures designs in which outcomes are determined by comparing pre–post differences in symptomatology or functioning. Then, at the aggregate level, the change over time is calculated using paired t-tests between T1 and T2, repeated measures analysis of variance or covariance, or generalized estimating equations (Wyrwich & Wolinsky, 2000). However, the sensitivity to change is directly affected by the sample size and, thus, can mislead to false conclusions. For example, small changes between T1 and T2, which might actually not be clinically meaningful, might reach statistical significance due to the large sample size. In contrast, it is possible to obtain a non-significant test because of small sample characteristics rather than lack of sensitivity to change. Another important disadvantage of the aggregate data analysis is that they are not useful for sufficiently determining the dynamic of positive and negative changes in the individual case and, as a result, meaningful individual changes taking place over the course of treatment are often neglected or overlooked. Individual changes are significant for both clinical practice and research questions since they address relevant questions such as treatment progress, responsiveness to interventions and further treatment necessity (Wyrwich & Wolinsky, 2000; Hays et al., 2005). A single change might be not significant at a group level but might be of value to the individual patient.
Many studies reveal that unchangeable static variables, such as prior offending history and membership in high-risk demographic subgroups, consistently predict recidivism (Lattimore, Macdonald, Piquero, Linster, & Visher, 2004; Trulson, DeLisi, & Marquart, 2011). Recent risk assessment research focuses on dynamic attributes — attitudes, values, and interpersonal skills that are modified by new experiences and, thus, may change during the residential stay (Labrecque, Smith, Lovins, & Latessa, 2014; McGrath & Thompson, 2012; Baglivio, Wolff, Piquero, Howell & Greenwald, 2017). The dissertation project aimed to expand on prior work by examining one specific dynamic factor/criminogenic need – hostile cognitions – as well as their possible change over time. As offenders' treatment programs are constantly subjected to assessment of treatment efficacy, studies are needed to identify if key therapy components such as antisocial/hostile cognitions contribute to a positive treatment outcome when deliberately altered. Thus, we examined changes within institutional placement as those are the highest risk youths, who arguably display the most intensely distorted cognitions.
To our knowledge, no study to date has used multiple methods to assess HAB as well as multiple statistical approaches to determine which method is most sensitive to changes in the treatment of distorted cognitions. Therefore, the purposes of this project are: (1) to evaluate the sensitivity to change of three semantically different tools assessing hostility-prone cognitions (structured objective questionnaires, vignettes as a semi-projective story complementation tool and a computer-based cognitive perception task) using three statistical methods for identifying aggregate change over time (group effect size ES, Cohen's d and standardized response mean SRM) and three statistical methods for identifying individual changes over time (the reliable change index RCI, individual effect size ES and standardized error of measurements SEM); and (2) to determine which of the three assessment tools proves most sensitive to capture changes in HAB post psycho-therapeutic treatment.
In theory, when measuring time-stable personal characteristics (traits) there is no or a little change expected, while tools, measuring states, should ideally be highly sensitive and variable (Krauth, 1995). Hence, we suppose that the structured objective questionnaires measuring traits, will show no statistically reliable change over time. In contrast, the semi projective vignettes are designed to assess interpretation biases and distorted beliefs, theoretically changeable through diverse cognitive-behavioral programs and social skills trainings. Therefore, we expect that the vignettes tool will be more sensitive to changes, since the construct it measures is dynamic in nature. In regard to the experimental task assessing the deficient social information processing, we expect no or a little change post treatment. The social-cognition experiment is designed to capture direct, immediate reactions, using a primary spontaneous response to briefly presented stimuli (see Methods). Indeed, there is some evidence addressing the modi?ability of the impaired recognition of affective facial expressions and the correction of this perceptual insensitivity with a brief implicit training approach (sensitivity to emotional expressions training, SEE training: Schönenberg, Christian, Gaußer, Mayer, Hautzinger, & Jusyte, 2014). However, without being intentionally intervened, perceptual deficits usually do not better over time. Since this is not the case in the current study, we do not expect a statistically reliable change over time to be measured by the experimental task.
Due to the different approaches toward the assessment of HAB as well as the particular goals of each individual study, the samplings, the measures for hostility and the statistical analysis varied across the three studies. Below, each study's method is described individually in order to provide a clear overview about the research approach.
As a common procedure in all studies, participants signed a written informed consent for their participation, including information about the purpose of the research, participants' right to decline to participate and to withdraw from the research, the lack of consequences of declining or withdrawing, confidentiality, data management, incentives for participation, and whom to contact for questions about the research and research participants' rights. After signing a written informed consent for their participation, the subjects were given verbal instructions describing the task procedures. After completion of the study, participants were debriefed and provided with an opportunity to obtain information about the results and conclusions of the research. Remuneration was provided to appropriately compensate subjects for their time, efforts, and inconvenience of participation. Even if subjects decided to withdraw from the study, they were compensated. To avoid undue influence and profit-related participation, the level of compensation was set to match subjects' economic status, resources, and minimal hourly payment rate in the state of Berlin (10 Euro).
Study 1.
Sampling and procedure
The starting sample was comprised of all 45 male adolescent violent offenders reported and subsequently convicted for a violent or sexual offense and currently receiving individual and group psychotherapy treatment in the Social Therapy Department (STD) of a German correctional facility for juvenile offenders in Berlin (Jugendstrafanstalt Berlin). At the time of the index offense, participants ranged in age from 17 to 24 years (M = 20.5 years, SD = 1.8 year). Almost half were German (48.9%, n = 22); 42% of subjects (n = 19) had a foreign background, of which 26.5% (n = 12) identified as Muslim, 4.4% (n = 2) were Polish, and 11.1% (n = 5) were Eastern European (Serbia, Bosnia, Latvia, Greece). Four participants (8.9%) had an unknown nationality or were considered stateless, that is, not recognized as a citizen of any country. The length of imprisonment ranged between 24 and 78 months (M = 42.9 months, SD = 14.2 months). With respect to the highest educational level at the time of the current offense, inmates did not possess a completed school degree in 60% of the cases, had an elementary school degree in 33.3% of the cases and possessed a high school degree in 6.7% of the cases.
In case of inclusion of previously committed offenses or a combination of several single registered crimes, we coded the most severe criminal offense. In accordance with the conceptual guidelines of the STD of the juvenile correction center, all current crimes were classified as violent crimes (including sexually motivated violent crimes). Consequently, the majority of the offenders were convicted of robbery, extortion, blackmail (n=18, 40.0%), followed by assault or other nonsexual violent offenses (n=12, 26.7%), (attempted) homicide (n=5, 11.1%), (attempted) murder (n=3, 6.7%), sexual violent offense (n=3, 6.7%), other offense (n=2, 4.4%), sexual abuse of children or juveniles (n=1, 2.2%) and fraud (n=1, 2.2%). All respondents except one had been convicted previously. The majority were convicted for a crime similar or equal to the current offense (80%, n = 36).
Measures for hostility
The five hypothetical vignettes used in study 1 were originally designed to assess hostile attributions in both institutional and noninstitutional social situations. The development and validation of the presented vignettes was described in detail elsewhere (Seager, 2005; Serin, 1991; Vitale et al., 2005). In short, all vignettes represented a social scenario in which imaginary provocateurs acted in an ambiguous way, and their actions resulted in a negative outcome for the subject of the story. The participants were asked to imagine themselves as subjects in those scenarios and explain why the provocateurs acted the way they did and how the participants perceived the provocateurs' behavior. The participants completed an identical questionnaire for each vignette that contained closed- and open-ended questions. We adapted the question set, developed by Serin (1991) and used in multiple studies (Seager, 2005; Vitale et al., 2005). It contained three main questions directly addressing subject's attributions of intent, and four secondary questions constructed to assess possible negative (not necessarily hostile) responses to differentiate between general negative and negative hostile attributions. The three main questions were as follows:
1. What do you think is the most plausible explanation of this situation?
2. How would you describe the provocateur in the story?
3. Did he/she acted intentionally, or it was an accident?
The four secondary questions were as follows:
1. Do you think his/her/their actions were accidental?
2. Do you think his/her/their actions were intentional?
3. Do you think his/her/their actions were done out of disrespect?
4. Do you think he/she/they meant to harm you?
Responses were analyzed using the thematic analysis procedure described by Braun and Clarke (2006). First, the dataset was studied for common patterns and themes in participants' responses. The data were read carefully to identify meaningful units of text relevant to the research topic. Second, units of text addressing the same issue were put together into thematic categories and given provisional definitions. The same unit of text could be included in more than one category. Then, the data were systematically reviewed to ensure that a name, definition, and exhaustive set of data to support each category were identified. The thematic analysis resulted in 42 prime categories, which were grouped into three key themes in terms of attributions of intent and two themes regarding the components of hostility. The reliability of the themes was established by a second researcher who recoded 100% of the data with a high level of interrater reliability, ranging from substantial to almost perfect agreement (Cohen's Kappa k ranging between .638 and .810, all ps .05).
Tests to see whether the data met the assumption of collinearity indicated that multicollinearity was not a concern (AQ hostility score: Tolerance = .77, Variance Inflation Factor (VIF) = 1.29; PSSI hostility score: Tolerance = .77, VIF = 1.29).
Stimuli and Experimental Task
The development and validation of the stimuli are already precisely defined elsewhere (Lobmaier & Perrett, 2011; Lobmaier et al., 2008). The acquired images represented faces of two male and two female actors varying in their emotional expression (angry, fearful, neutral, or happy) and viewing angles (direct gaze direction 0° and rotated gaze directions to 2°, 4°, 6°, and 8° to the left and right). Gaze and head orientation were overlapping. Lobmaier and colleagues (Lobmaier & Perrett, 2011; Lobmaier et al., 2008) validated the set of stimuli showing correct recognition and classification of the emotional expressions. A series of example images for each emotional expression in different viewing angles is shown in Figure 1. Using a forced-choice, yes-or-no task, participants decided whether a presented face was looking directly at them or not. Each stimulus was presented for 300 ms on the screen and participants submitted their answer by pressing the j key for yes, looking at me and n key for no, not looking at me. All stimuli were presented in random order. Participants received no feedback on whether their decision was right or wrong. After participants had answered, the next trial appeared automatically on the screen.
First, we separately calculated the means for both looking-at-me and not-looking-at me answers for each emotional expression and viewing angle for each participant, and then transformed the means in percentages. A higher mean indicates that a person feels strongly looked at, while a lower mean reflects a weaker self-referential perception in the particular gaze direction. Corresponding gaze directions to the left and to the right were merged together because prior analyses showed no significant difference in the effects between left and right direction (Lobmaier & Perrett, 2011; Lobmaier et al., 2008).
Second, to examine the hypothesis of a positive self-referential bias, looking-at-me answers (dependent variable) were subjected to a two-way repeated measures analysis of variance (ANOVA) with the factors emotion (neutral, angry, fearful, happy) and gaze direction (0°, 2°, 4°, 6°, 8°) while controlling for the effects of age, educational level, and delict. Then we added the individual score of self-reported hostility as a covariate to estimate the potential influence of hostility in gaze perception. The effects of age, educational level, and delict were also controlled for in the multivariable model.
In a third step, the effects of hostility on response latencies were analyzed repeating the same statistical steps. First, we examined the looking-at-me answers with a two-way repeated measures ANOVA with the factors emotion (neutral, angry, fearful, and happy) and gaze direction (0°, 2°, 4°, 6°, 8°). Then we added the self-reported hostility scores as covariate to assess the effect of hostility on the response times.
We created an additional exclusion criterion based on the level of performance in the experiment and the frequency of yes/no responses. Participants who completed less than 100% of all trials (288 in total) and exhibited a response bias by answering with a yes or no in more than 80% of all trials were excluded from the final analyses (n = 18).
Partial eta square (?p 2) is reported to indicate the effect size. In case of violating the sphericity assumption, we used the Greenhouse-Geisser correction. Significant main effects were followed by Bonferroni corrected pairwise comparisons. Post hoc analyses of significant interactions included separate one-way ANOVAs for repeated measures. Statistical analyses were calculated using SPSS version 22. The significance level for all tests was p .050; see Table 2).
In the final sample (N=18), the subjects ranged in age from 16 to 22 years (M = 19.5 years, SD = 1.82 year). The majority had an immigration background (61.1%, n = 11), of which 33.4% (n = 6) of Arabic origins, and each 5.6% (n = 1) of Polish, Bosnian and Greek origins. The rest (38.9%, n = 7) were Germans. Two participants (11.1%) had an unclear or unknown nationality or were considered stateless. The length of imprisonment ranged between 24 and 78 months (M = 42.9 months, SD = 13.9 months). All participants had been convicted previously, the majority of the subjects for a crime similar or equal to the current offense (50%, n=9). With respect to the highest educational level at the time of the current offense, more than the half of all inmates (61.1%, n = 11) did not possess a completed school certificate and 38.9% (n = 7) had a main or primary school degree. Accordingly, none of subjects were qualified for any profession. With respect to the professional situation of the inmates at the time of the index offense, approximately two-thirds (66.7%, n = 12) were unemployed, each 11.1% (n = 2) worked either as general workers or casual workers and each 5.6% (n = 1) had a personal business or an unclear professional background. The majority of subjects (77.8%, n = 14) were unmarried, 22.2% (n = 4) were in partnerships. Regarding the convicted offense, the majority of the inmates were incarcerated for robbery and extortion (44.4%), followed by nonsexual violent offenses such as assault (27.8%), (attempted) homicide (11.1%), (attempted) murder (5.6%), sexual violent offenses such as rape (5.6%), and fraud (5.6%).
Identically with study 2, three different approaches to assess hostility were applied: two self-report scales (BPAQ and PSSI), one semi-projective technique (vignettes) and a computer-based experimental perception task. In the current sample, for BPAQ we found an acceptable internal consistency (Cronbach's ? = .75), identical to the one found in the validation study (Cronbach's ? = .75, Herzberg, 2003). For PSSI, we found an acceptable internal consistency (Cronbach's ? = .74), almost identical with the results of the original validation study (Cronbach's ? = .79, Kuhl & Kazén, 2009). For the vignettes tool, we found a good internal consistency (Cronbach's ? = .87).
The experimental task was conducted and analyzed on the same way like in study 2.
Treatment/intervention characteristics.
The Social Therapy Department of the Juvenile correction center focuses on offender treatment (systemic, behavioral and analytical orientation) based on the RNR model and, concretely, on the determined dynamic risk factors (Andrews ; Bonta, 2010; Gendreau, 1996). As per German juvenile penal law (JStVollzG), the Social Therapy Department was designed to host and provide psychological treatment for three main offender groups: (1) severe violent offenders with a high recidivism risk, (2) severe sex offenders with a high recidivism risk, and (3) persistent "chronic" offenders. A probation period of three months serves as an adaptation time to the treatment, exploration of the treatment motivation and adjustment to the Social Therapy Department rules, norms and principles, as well as verification of the effective treatment interventions. The actual therapy phase lasts twelve months. However, the treatment duration and intervention length both depend on the individual case and are adapted to the particular needs and risk factors of each inmate. In addition to the individual treatment plan, of central importance is the mandatory participation in the program Reasoning ; Rehabilitation by Ross, Fabiano and Ross (1986). This is a cognitive-behavioral group program developed for the acquisition and training of cognitive and social skills. The interventions aim to provide the offenders with alternative behavioral and emotion regulation strategies, and to change distorted cognitions in order to ensure successful reintegration into a stable social environment, including suitable housing and employment opportunities.
Statistical analyses for the sensitivity to change for all measures
The study examined the dynamics in how aggregate and individual hostility scores changed over time, and precisely determined the sensitivity to change of all assessment measures. In a simple pretest-posttest design with one intervention group hostility assessment scores from Time 1 (pretest) and Time 2 (post-test) were compared utilizing a paired t-test. During this period, all participants were subjected to individual and group cognitive-behavioral treatment.
At the aggregate level, we computed the overall score for each hostility scale at T1 and T2. Then, the statistical signi?cance of the mean value differences from T1 to T2 was assessed using t-tests for paired samples. We carried out three statistical approaches for calculating the changes at the aggregate level. First, Cohen's d was used to measure the effect size. Cohen's d was calculated as follows:
d=((M_t1- M_t2))/?SD?_pooled where ?SD?_pooled=?((?SD?_1^2+?SD?_(2 )^2)/2 ) =?(((n_1-1) S_1^2+(n_2-1) S_2^2 )/(n_1+n_2-2)) (1)
Then, standardized effect size ES was used as second approach. To calculate the ES, the difference in the means between the measurement points (e.g. between baseline and follow-up) is divided by a measure of variance (e.g. the standard deviation at the baseline):
ES=((M_t1-M_t2))/?SD?_t1 (2)
The standardized response mean (SRM) with the standard deviation of the values' difference was used as a third effect size indicator for assessing aggregate changes during treatment (Igl, Zwingmann, & Faller, 2005). SRM was computed by dividing the mean score change (i.e. baseline minus follow-up) by the standard deviation of the change scores as follows (Igl et al., 2005; Küfner, Buchholz, Rist, Kraus, & Lindenmayer, 2009):
SRM= ((M_t1-M_t2))/?SD?_difference where ?SD?_difference= ?((?SD?_1^2)/n_1 +(?SD?_2^2)/n_2 ) (3)
Our choice of sensitivity to change estimates was based on our assumption that change would be homogeneous across the sample meaning that the sample consists of patients who are expected to change by approximately the same amount over the study period (Stratford & Riddle, 2005). Thus, distribution-based statistics that assume homogeneous change, including Cohen's d, effect size (ES), standardized response mean (SRM), and paired samples t-test (Stratford ; Riddle, 2005), were computed for the total sample. Cohen's d, ES and SRM express change scores in terms of the underlying sampling distribution, using standard deviation estimates. All three coefficients are standardized indicators of power of an instrument to detect true change, with larger values indicating higher sensitivity to change. Belonging to one "statistical family" (Stratford & Riddle, 2005)., these indices are based on the same assumptions concerning the sample's change characteristic and, thus, results obtained through Cohen's d, ES and SRM must be corresponding to each other.
A major strength of these coefficients is their ability to be used to assess which measures are more responsive to change than others. On the other hand, all the measures discussed in this section share the limitation that they do not relate changes in the measure to corresponding changes on an external clinical status measure at the individual level. The statistics simply examine the extent of change in the measure over two occasions. This is problematic as a statistically significant change in the measure may occur without a corresponding change in clinical or health status. In other words, the observed change in the measure may not reflect important change in the condition of the patient (Husted, Cook, Farewell, & Gladman, 2000). To counteract this weakness, we calculated individual change indices.
Similarly, we carried out three approaches for calculating the individual clinically meaningful changes over time. First, a standardized effect size for individuals (ES) was computed as the difference between each individual's pre- and post-test scores, divided by the group standard deviation at T1, SD1 (Eisen, Ranganathan, Seal, ; Spiro III, 2007)
ES=(x_pretest-y_posttest)/s_pretest (4)
The advantage of this approach is that the changes, found in different samples and in different scales, are comparable. The approach has the disadvantage that the reliability is not taken into account. Based on their individual ES, participants were then classified in three groups: a medium to large improvement (ES ; 0.50), no effect or a small effect (-0.49 to 0.49), or a large decline (ES 0.50 was selected based on a review of Norman and colleagues (Norman, Sloan, ; Wyrwich, 2003) who found that the mean minimally important difference was almost exactly equal to an effect size of 0.50.
Second, we computed the reliable change index RCI (Nachtigall ; Suhl, 2002) as follows: The posttreatment score is subtracted from the pretreatment score and divided by the standard error of the differences.
RCI= (x_pretest-y_posttest)/(s_x ?(2*(1-r_xx ) )) (5)
Study participants were then placed into three groups based on their RCI scores: reliable decline (RCI 1.96).
Third, we computed the standard error of measurement SEM (Hays et al., 2005). The SEM was strongly recommended as a useful statistic for assessing individual changes in numerous medical, cognitive, and behavioral conditions (Hays et al., 2005). The SEM was computed as follows:
SEM= s_1 ?(1-r_xx ) (6)
where S1 is the standard deviation at T1 and rxx is the internal consistency reliability coef?cient at T1. Recent research has reported that one SEM consistently corresponded to a minimal clinically important intra-individual change (Wyrwich, Tierney, ; Wolinsky, 1999). Consequently, we also used one SEM as the criterion for clinically meaningful change.
Participants were then grouped as declined if their T1-T2 difference scores decreased by at least one SEM, stable if their T1-T2 difference scores were less than one SEM, and improved if their T1-T2 difference scores increased by at least one SEM.
The purpose of the greater project was go investigate the sensitivity to change of three different hostility measures. The studies analyzed, for the first time, quantitative, qualitative and experimental data in a sufficient and systematic way by including tools originating from different theoretical and empirical paradigms. Therefore, three approaches to the assessment of hostility and its change during treatment were applied. The first study focused on a qualitative assessment of participant's social perception, examining how people perceive ambiguous situations and if they relate interpretations of behavior of others as directed toward the self. Based on the combined evidence from both the spheres social cognition and abnormal processing of gaze cues in social interactions, the second study was designed as an experiment on perception biases, processing deficits and biased hostility-driven interpretation of gaze direction. The overall project set the assessment tools in time perspective and analyzed tools' capacity to measure changes occurring during a one-year of cognitive-behavioral treatment (CBT) using a multi-method assessment. Determining the sensitivity to change on both aggregate and individual levels, the study aimed to find out which assessment tool is most sensitive to changes in the treatment of distorted cognitions.
The purpose of the first study was to uncover the ways people perceive and interpret ambiguous social situations with a negative outcome. The vignette method was used as a qualitative questionnaire and analyzed through a thematic analysis. We identified three themes (positive, negative, and neutral) addressing how participants perceived and interpreted ambiguous social situations and provocateurs' behavior and two themes regarding the components of hostility (provocateur-related personality features and the relation between the participant and the provocateur). Thus, although the vignettes were originally developed to detect hostility-prone perception bias and hostile attributions of intent (Dodge, 1980), they have potential to reveal a wider set of different attributions of intent, both positive and negative. Thus, vignettes were not limited to assessment of hostility specifically.
In regard to hostility, we found that it wasn't reflected by a plain negative attribution of intent. On the contrary, the construct represented a more complex phenomenon consisting of the following three components: (1) a bad personal character/temper of the provocateur, (2) provocateur's actions driven by malevolence and deliberate intention to harm, and (3) hostility particularly directed toward the participant. Thus, the provocateur's character was not only perceived as generally evil or low, but also as evil or low in the concrete social situation toward the specific disadvantaged person. Hostility was explicitly linked to an interpersonal context in which the participant felt taken advantage of, used, underprivileged, or involved in an unequal power ratio. Hostility didn't appear as a context-independent generalized attribution but much rather as an individualized participant-related perception bias of an ambiguous situation.
Consistent with prior studies on dyadic relations (e.g., Coie et al., 1999; Hubbard, Dodge, Cillessen, Coie, & Schwartz, 2001; Simpkins & Parke, 2002) and emotional valence of the relationship between two individuals (Lemerise & Arsenio, 2000), our data analyses supported that in addition to personal dispositions (i.e., provocateur's effects), there is one more factor—familiarity with the provocateur—that seemed to moderate the processing patterns, interpretation of ambiguous social cues, and attribution of hostile intent. Participants were inclined to attribute higher hostility to acquainted associates than they did to unfamiliar or unknown peers, implying that subjects might be prone to interpret the behavior of known peers in ways that sustain already present opinions or judgement of these peers. In contrast, an interaction with an unknown partner seemed to inhibit the attribution of hostile intent, presumably due to the absence of prior experience with the concrete person. This finding suggests that despite the existence of a friendly relationship, when in doubt juvenile offenders are predisposed to perceive the ambiguous actions of their friends, first and foremost, as intentional and hostile, possibly resulting from previous negative interactions with these peers. However, based on past experience, juveniles may hold unfairly negative views of their acquaintances and ignore or discount their positive characteristics and, thus, search for and interpret evidence selectively to reinforce current beliefs or attitudes (Fine, 2006). When it comes to judging an unknown person, juveniles tend to be more cautious and reserved. Thus, different relationships with known versus unknown people might explain the two different interpretation processes.
Furthermore, we found that hostile attributions of intent intensified in the presence of a power differential between the participants and the interacting partner (Vignette 5). The power differential is the enhanced amount of role power that accompanies any position of authority and represents the influence that empowered people have as compared with people in low-power positions (Dahl, 1957). When involved in a social scenario, in which others have authority over an individual in a more dependent vulnerable position, all participants took intuitively the perspective of a subordinate figure and tended to interpret the social situation from a disadvantaged and suppressed power position. Assuming an asymmetric power status seemed to heavily impact the interpretation of ambiguous social cues as hostile and to attribute hostile intent of behavior of others.
In addition, the perception and interpretation of the ambiguous situation and behavior of others seemed to be dependent on age, ethnic origins and conduct style. While younger participants were inclined to generate more negative interpretations, older participants tended to think in more relative terms, redefine ambivalence in a positive way, or also interpret the social situation in a neutral abstract way. Participants with a Muslim cultural background were more likely to provide negative interpretations, while positive or neutral attribution of intent seemed to be more prominent in the German ethnic group. A tendency for more frequent violent behavior and misconduct during imprisonment was also associated with negative interpretation bias. First, these results suggest that the cognitive-emotional development and maturing processes might moderate the differences in the ways individuals process ambiguous social cues with interpretations shifting from negative to positive and neutral when age progresses. Second, the differences in the attribution of intent might also depend on the individual's origin and socialization context, suggesting that there are certain ethnic-related characteristics that would make an individual more likely to interpret ambiguous situations in negative, positive, or neutral way. Third, the association between negative interpretations and frequency of misconduct reveals a potential link between negative (hostile attributions) and reactive aggressive behavior. Determining the different factors leading individuals to demonstrate specific positive or negative attributions is particularly relevant in developing best treatment strategies for juvenile offenders as well as identifying certain characteristics that might form individual's perception and interpretation bias. Additional research might focus on potential variables that might influence the way individuals view ambiguous social situations.
In the second study we extended the findings of hostility as a self-related perception bias, obtained through the qualitative analysis, to the field of social perception. To test if HAB would emerge instantaneously as a reaction within an ambiguous social situation, we created a computer-based experimental task simulating social interaction. Based on previous empirical findings and theoretical considerations in the spheres of social cognition in inmates, we considered both the factors eye gaze and facial expression crucial for creating the visual illusion of being looked at. Thus, the purpose of the second study was to investigate both the self-directed perception of (averted) eye gaze and facial affect processing in incarcerated juvenile violent offenders with high levels of hostility. A holistic assessment of hostility was conducted, including measures defining hostility as a component of the aggressive behavior, as a personality trait, and as attribution and interpretation bias in ambiguous social settings.
In accordance with previous research (e.g., Lobmaier et al., 2008), our results indicated that the emotional facial expression influenced the frequency of perceived gaze as self-directed. In comparison to faces with negative expressions such as fear and anger, happy faces evoked a stronger feeling of being looked at. Further, juvenile violent offenders also demonstrated an increased sensitivity to perceive happy faces fastest (as reflected by the RTs), followed by neutral, angry, and fearful faces. Amongst the negative facial expressions, angry faces were more likely to be perceived as self-directed than fearful faces. Regarding the viewing angle, participants tended to respond with more looking-at-me-answers by less averted gazes (0°, 2°) than by larger viewing angles (4°, 6°, 8°) which reflected a logical pattern of perceiving straight-on gaze accurately as self-directed. Against the expectations, RTs did not increase with the increment of the viewing angle but showed a dynamic fluctuation: RTs were shortest for stimuli facing the observer in the 0° condition, followed by 6°, 2°, 8°, and 4°, indicating a high ambiguity in the middle angles conditions.
There was also a significant interaction between both factors emotion and gaze direction. Emotion showed a significant influence of the gaze perception in the conditions of 2° and 6°, while there were no significant effects in the remaining viewing angles, suggesting that the emotional face expression plays an essential role of gaze processing, especially in highly ambiguous conditions (angles). While it might be easy to determine a direct gaze (0°) as self-directed and strongly averted gaze (8°) as turned aside, the middle viewing angles, such as 2° and 6°, seemed to evoke a high uncertainty in deciding whether a face is looking at the observer or not. In such ambiguous conditions, participants not only tended to perceive gazes as self-directed but were also faster when judging perceived direction (as reflected by the RTs). Moreover, the effect of the facial expressions of the self-referential perception of averted gazes seemed to be specific for happy faces: compared with other emotional expressions participants tended to perceive averted gaze of happy faces in all gaze directions as more self-referred.
When adding hostility as a covariate in the multivariable analysis in order to investigate the relationship between hostility and the self-referential gaze perception both the factors, emotion and gaze perception, their interaction, as well as all interactions comprising the factor hostility failed to reach statistical significance (all ps ; .050). Contrary to expectations, participants with a high level of hostility neither exhibit a preferable processing of angry faces nor tended to perceive angry faces as looking at the observer significantly more often than faces with neutral or positive expressions. In highly ambiguous conditions, such as 2°- and 6°-averted gaze directions, participants tended to perceive happy faces as self-directed.
However, preferred processing of happy faces could also be an indication of a hostile reaction to provocation in social situations. Hostility and aggressive behavior are both triggered by the belief that the individual has been threatened or provoked (Dodge, 2006). Usually, hostile response bias to emotional faces is shown to occur as a reaction to stimuli containing proportions of an angry expression (Schonenberg ; Jusyte, 2014). However, similarly to faces expressing negative emotions, happy faces might also signalize threat in hostility-prone individuals. Although smiling faces are usually associated with a positive and, thus, nonthreatening approach to another person, they could also imply an insulting or offensive attitude and, therefore, be perceived as disturbing or unpleasant and cause feelings of annoyance, distress, or irritation. Future research on the possible link between a positive perception bias for happy faces and hostility is necessary to expand our knowledge of the deficiencies in the processing of social ambiguous cues.
The lack of a statistically significant correlation between hostility-prone attributions and a preferable processing of angry face gazes as self-directed might be partly explained by empirical evidence showing impaired recognition of unambiguous facial affect among antisocial populations (e.g., Marsh ; Blair, 2008). Logically, before perceiving the gaze of a face as self-directed, one must first be able to detect, identify, and correctly categorize the onset of the facial affect. The assumption of an increased sensitivity to threatening or provoking faces in aggressive offenders can be considered counterintuitive given the fact that there is no consistent evidence showing prioritized detection of angry facial cues in those with antisocial tendencies. In fact, some recent findings provide grounds to assume that aggressive individuals show reduced, rather than increased, sensitivity to facial signals of anger (Best, Williams, ; Coccaro, 2002; Fairchild, Stobbe, Van Goozen, Calder, ; Goodyer, 2010; Fairchild et al., 2009; Schonenberg, Louis, Mayer, ; Jusyte, 2013). Such deficits in facial affect recognition can at least partly explain the lack of a consistent link between hostility and biased perception. Reduced sensitivity to angry expression might block the continuity of the perception– interpretation process and, thus, interrupt the potential manifestation of a biased interpretation and attribution of intent.
In sum, evidence from previous studies suggests that aggressive individuals tend to habitually attribute hostile intent to others in social situations with high ambiguity (Dodge, 2006). In addition, this characteristic pattern is not only driven by concrete actions of a social partner that might be misinterpreted as deliberate provocation but also emerges in the processing of facial cues (Schonenberg ; Jusyte, 2014). However, this deficit processing does not seem to take place on a self-referential basis. Our data support past evidence suggesting that antisocial individuals show reduced sensitivity to negative (angry) facial expressions (perceptual bias), but we could not confirm the hypothesized link between hostility and self-referential gaze perception (cognitive interpretation bias).
In the final dissertation project, we used a multi-method approach for assessing hostility-prone cognitions, combining and analyzing data obtained through structured questionnaires, a semi-projective tool and experimental data. The focus of the study was to compare the tools' capacity to measure changes over time for which we used three methods of determining the sensitivity to change on both aggregate and individual level. Several major points emerged from the analysis.
Consistent with the results at the aggregate level, the vignettes method identified the highest proportion for clinically meaningful change, suggesting large improvement (decreasing values) of the hostility-prone cognitions over course of treatment. While the vignettes as a semi-protective method were the only assessment instrument that showed sensitive to change at the aggregate level, at the individual level all three assessment methods showed able to measure change. However, individual change assessed through both the structured self-report questionnaires was inconsistent indicating somewhat contradictory results. As assessed through AQ, a questionnaire defining hostility as a feature of the aggressive behavior, the change was predominately negative suggesting that the tendency to perceive and interpret signals in a hostile manner increased over time. On the other hand, PSSI, a questionnaire measuring hostility as a personality feature, showed clearly positive change over time indicating that the hostility-prone cognitions reduced over the time of treatment. Thus, AQ suggested deterioration, while PSSI suggested an improvement of HAB.
This apparent contradiction is likely to be accounted for by the different theoretical approaches underlying the concept of hostility-prone cognitions. While AQ assesses global, gist-based judgments of hostility, PSSI focuses more on the individualized, personalized beliefs about HAB. According to the RNR-model for effective correctional practices, crime supportive dynamic factors such as antisocial cognitions can be consistently modified when the interventions applied are adapted to target individual criminogenic needs. Since HAB as measured by PSSI represents a personalized attribution style and reflects the extent to which individuals identified with hostile beliefs, it might have been more responsive to treatment and, as a result, showed a reduction over time. It seems that changes in hostility-prone cognitions as measured by PSSI reflected the extent to which adolescents identified with hostile beliefs before and after the treatment. The definition of HAB as a personal attribution style suggests that HAB is conditional and as life conditions change – through natural grow, maturity, treatment – HAB too priorities positive change. In contrast, measures that assessed global judgments of hostility, such as AQ, might be more robust to improvement (positive change) because they reflect the core of a general belief that might be stable, universal and inflexible to intervene. Paradoxically, such beliefs might even show a negative change, indicating aggravation and worsening due to striving to preserve existing beliefs and internalized values. Hence, AQ might have shown negative change as a protective rather than reflective relation.
These results support a dual-processes interpretation of the dynamics in HAB according to which observed changes depend on whether the cues in questions trigger a specific (personal) or global (general) processing. Paradoxically, changes in HAB are sometimes related positively and sometimes related negatively to treatment. This difference might be impacted by the construct's level of aggregation (or generalization). Both the structured measures for hostility were not on a comparable level of generalization which could have impacted the findings and produced ambiguous results. These inconsistent findings underscore the need for a better theoretical grasp of the circumstances that produce positive or negative change and call for future research.
In comparison with the other two approaches (structured questionnaires and experimental task), the vignettes were sensitive to identifying the largest proportion of positive, clinically meaningful, improvement (individual change). Interestingly, even the experimental perception task showed sensitivity to change. The perception and processing of happy faces seemed to be most dynamic and fluctuating, indicating the highest proportion for clinically meaningful change (followed by neutral, angry and fearful). While the change, indicated by the perception of happy and fearful faces, seemed random, the perception of angry and neutral faces indicated clearly a negative change i.e. increasing values from T1 to T2. These results suggest that participants showed a preferred processing of angry and neutral faces as self-related at T2, and thus, a stronger perception bias in comparison to T1. The results for the self-referential perception of gaze directions showed the same pattern. At the individual level, across all viewing angles, the tendency to perceive averted gazes as self-related slightly increased from T1 to T2, as reflected by ES, RCI and SEM.
Consistent with previous ?ndings reported in the literature, both the SEM and ES methods identi?ed a higher proportion of individuals as meaningfully improved or declined than did the RCI method (Hays et al., 2005). This data supports Wyrwich and Wolinsky's assumption that the SEM may be a better method for determining meaningful change than ES and RCI (Wyrwich & Wolinsky, 2000). While the ES uses the sample dependent standard deviation, the SEM does not vary from sample to sample, and, thus, the SEM provides a more stable method for determining clinically meaningful change (Wyrwich & Wolinsky, 2000). These results are also consistent with the suggestion that the RCI may be too stringent a criterion for determining clinically meaningful change (Jacobson, Roberts, Berns, & McGlinchey, 1999).
Limitations of the dissertational study
The dissertational study has a couple of sample-related limitations. Whether this dynamic in the change over time is typical for juvenile offenders in general, for the type of their treatment or for the institution of treatment, cannot be determined empirically-statistically without a corresponding control or comparison group. Hence, we do not know the extent to which the results can be generalized. However, the focus of this study was to use a multi-method approach for assessing cognitive and interpretation biases in a single-group pretest-posttest experimental design and to measure the degree of change occurring as a result of treatments or interventions at both aggregate and individual level, rather to compare different study groups. Future studies might consider using a two- (or multi-) group design in order to compare potential differences between intervention and control groups.
Further, the sample was very highly specified and included multiple selection criteria: age (juvenile and adolescent), offense of conviction (violent and sexual crimes only), current incarceration, current psychotherapeutic treatment. Strictly including only participants who fulfilled all criteria resulted to a small initial sample size at T1. Nevertheless, in case of investigating a rare phenomenon or studying a highly specified sample an underpowered study is justified. Individuals most at risk such as violent and sexual offenders in treatment are by definition small in number when compared to the larger, dominant population (e.g., offenders with no need of treatment). Particularly when considering high-risk behaviors such as recurrent violence, forensic researchers are often limited by small populations and, respectively, by small numbers of potential participants. It could be argued that studies using small sample sizes are not meant to quantify general performance within a population but merely to document the existence of an effect. Thus, small sample studies may not always find statistical significance, but the effect may be clinically important. Yet, small sample research is critically important as the research questions posed in small samples often represent serious phenomena in vulnerable and underrepresented populations. Although the findings of the study cannot be generalized and are not necessarily applicable to all, they must be appreciated in their specific nature and importance in understanding the unique experiences of the individuals of interest. While presenting challenges for current methods, it is important to note that small sample research can also expand our horizons. New challenges require innovative solutions. Innovation is the foundation of scientific advances.
A more relevant sample-related limitation is represented by the very large drop-out rate between both the assessment times T1 and T2 that caused a severe loss to follow-up (27 of 45 participants) and naturally reduced the initial sample size by 60%, which may have resulted in overestimation of effect sizes and limited reproducibility and generalizability of the results. Thus, results may not be sufficiently powered to detect a more precise difference between T1 and T2 as measured by some of the assessment methods. In addition, the large attrition rate could have put a bias on the result of the study. As shown in Table 1, most of the reasons for drop out were motivated by major positive or negative changes in the participants' conduct. Similar numbers of subjects dropped out because of completing of the full-term of imprisonment (possibly associated with positive response to treatment or simply related to the usual course of imprisonment for various offenses), early termination of parole due to positive conduct (theoretically associated with positive response to treatment), and relocation to regular imprisonment due to negative conduct (theoretically associated with negative response to treatment). These major reasons for drop-out might each have different impacts on the findings if the subjects had remained in the study, which makes it challenging to determine whether the composition of the final participant pool was reflective of the population of interest. On the other hand, it can be argued that as the positive and negative reasons for drop-out were to a balanced proportion, it did not show preference to either which in an overall reduced sample size at T2 and, hence, had no impeding effect. Since positive (n = 8) and negative (n = 9) reasons for drop-out were in an almost identical quota, the reduced sample size at T2 constituted and corresponded in ratio to the same sample characteristics at T1.
One more factor influencing the evaluation of individual changes is the reliability of the assessment tools. Ideally, individual changes should be considered only when the reliability is 0.90 or more. However, this level of reliability is rarely achieved in clinical practice. Recent work has suggested that this reliability level may be too stringent and that although caution should be exercised when interpreting results of assessments with less than optimal reliability, use of individual assessment data can still be valuable (Hays et al., 2005). In addition to this, the problem with the reliability of (semi-) projective techniques is well known. The practice of using projective techniques for the measurement of personality change, such as in evaluating the effects of psychotherapy, ideally rest on the assumption of substantial retest reliability. If the retest reliability is good, then the changes occurring between T1 and T2 are attributed to the influence of external factors, such as intervention or treatment. However, the assumption of substantial retest reliability is rarely given, since projective techniques are highly sensitive to the slightest changes in the subjects, reflecting transient moods, etc. This makes it difficult to determine the nature of the change and creates a risk to misunderstand and falsely interpret a high error variance in the variability of the items as sensitivity to reflect real changes in the variable (Krauth, 1995). Since our study is the first one to test the sensitivity to change of the vignettes, we consider our findings sufficient to set up the base for future research in this area.
Another issue is represented by the facial expression/gaze direction perception task that focused on the sped-up stimulus exposition of faces. Participants were tested for their performance while categorizing viewing angles as self-related or averted. It is unclear to what extent individual differences in performance at both T1 and T2 are due to the difficulty of categorizing the gaze direction of an emotional expression and to what extent they are due to the sped-up nature of the task. Normally, under conditions of unlimited time, unimpaired subjects frequently perform at ceiling in perception tasks. In order to complicate the task difficulty, stimulus exposition can be sped up using brief exposition time which usually decreases accuracy rates but does not eliminate speed-related individual differences. Limited exposition times are likely to favor participants high in perceptual speed.
We found a slight tendency for the self-referential perception bias to increase form T1 to T2, meaning that at T2 participants showed a stronger preference to perceive and interpret averted gazes as straightly directed to oneself (over all four emotions). Nevertheless, this change did not appear at the aggregate level, and was only found in individual changes. Thus, the increment in the individual changes can be highly dependent on various external factors, such as different concentration levels and performance motivation at T1 and T2, individual's response bias, general cognitive ability, mental speed, layout of the physical environment etc. The influence of other variables cannot be rule out. Due to this, determining the sensitivity to change of a cognitive-perception task turned to be an exceptionally challenging task. Rather than a single, general mechanism that explains individual differences, a number of processes appear to be involved in the social-cognitive functioning of the individuals.
In recent years researches have repeatedly identified criminogenic needs as important to impacting criminal conduct and reducing reoffending (Andrews & Bonta, 2006). Since criminogenic needs have been shown to be variable, susceptible for modification and responsive to interventions, they are also known as dynamic factors indicating they ability to change and transform when targeted for change in rehabilitation programs.
When researches and clinical practitioners assess either general clinical constructs or specific forensic-related measures, they routinely collect diverse information (e.g., self-reports, objective instruments) to inform their assessments. Within forensic settings, the principles of risk, need, and responsivity should be reflected in offender assessment (Andrews et al., 1990). Here, it is argued that HOSTILITY might contribute to a more comprehensive and accurate assessment of recidivism risk as well as of factors empirically and theoretically associated with criminal conduct (e.g., HOSTILITY).
In treatment outcome studies, the ability of the applied instruments to detect significant change in the status of the dynamic factors over time is crucial and, vice versa, studies based on interventions proven effective are suitable for testing the sensitivity for change of different assessment methods.
The present research project in its entirety examined the sensitivity for change of diverse measures widely used to assess hostility-prone cognitions. While some of the measures required offenders to self-report the extent to which they identify with different hostility-driven beliefs, other measures assessed hostility as an interpretation bias to ambiguous social cues and a phenomenon of the interpersonal perception. Such multi-method assessments have several advantages. Using information from multiple tests rather than a single tool assessment allows clinicians to better guide diagnostic decisions and to gain a holistic systematic view of an individual's cognitive distortions. The assessment approach we propose in our project is a time efficient method to measure a wider range of manifestation of HAB than any single HAB-assessment method can measure separately. It is based on three foundational sources of information (i.e., practice, research and test development) that provide the knowledge necessary to organize theory-driven, comprehensive, reliable, and valid assessments of HAB and its change over time.
In this project, we applied different assessment tools of HAB among young violent offenders, and in particular tested them comparatively in terms of their sensitivity for change processes. Demonstrable change-sensitive diagnostic instruments for the detection of criminological needs are of very high value for forensic practice, for example for the assessment of treatment effects. However, they would also be of great scientific value because comparable approaches to the forensic field are currently lacking.
According to our overall results, the vignettes tool is most informative in terms of adequately assessment of the components of HAB as well as its change over time. When applied as a qualitative method, vignettes reflected the ways individuals constructed, perceived, interpreted and attributed intent to ambiguous social situations and how they expressed hostility in relation to the provocateur as well as to the self. Due to the incorporation of open-ended questions, vignettes were able to trigger individual cognitive processing patterns and evoke an in-depth, detailed analysis of HAB in its entirety. Thereby, data obtained through the vignettes can be considered very valuable to contributing to a better understanding of individuals' social realities and drawing attention to instructional perception processes, interpretation patterns, and social structural features. The evidence gained by exploring HAB qualitatively could be used as a basis for developing more relevant and specific treatments and rehabilitation strategies for individuals who perceived ambiguous social vignettes in a biased way. Thus, vignettes seemed to best correspond to the RNR-principles of effective correctional treatment in that they were able to reflect personal individual criminogenic needs (antisocial hostile cognitions) as expressed and defined by the individuals themselves.
When assessing change at the aggregate level vignettes were most informative in terms of detecting statistically significant change in the pretest-posttest comparison, representing a decrease in the mean values from T1 to T2. Consistent with these results, at the individual level the vignettes method also identified the highest proportion for clinically meaningful change, suggesting large improvement (decreasing values) of the hostility-prone cognitions over course of treatment. We suggest that the ability of the vignettes to reflect homogeneous changes are due to the character of the semi-protective technique. Projective techniques typically present respondents with an ambiguous stimulus and ask them to disambiguate it, or require participants to generate a response following open-ended instructions. In addition, most projective techniques permit respondents considerable flexibility in the nature and sometimes even the number of their responses (Lilienfeld, Wood, & Garb, 2000). Since individuals have various (criminogenic) needs which can be both conscious and unconscious, it seems plausible that these needs can be better triggered to reveal the thought process of a person when using an unstructured open-ended response semi-projective test permitting individual to spontaneously project their internal needs, than using a structured questionnaire that require responses to an explicitly asked questions. Thus, all the prevailing motivations, relevant beliefs and current attitudes are detectable quickly through a method such as the vignettes.
While treatment based on the RNR-principles progresses, internal (criminogenic) needs are prone to change. Interventions directed toward changing distorted or dysfunctional cognitions (cognitive restructuring) or teaching new cognitive skills include therapeutic techniques such as structured learning experiences designed to affect such cognitive processes as interpreting social cues, monitoring one's own thought processes, identifying and compensating for distortions and errors in thinking, reasoning about right and wrong behavior, generating alternative solutions, and making decisions about appropriate behavior. When cognitive biases, such as the tendency to perceive and interpret ambiguous social cues as hostile, get discovered, revealed and treated accordingly, then the whole mindset towards those stimuli gets modified. Thus, situations that once triggered hostility, might now be perceived as benign or neutral due to the intervened, once unconscious criminogenic need, and the new learned cognitive skills to deal with one's deficiencies.
Write MORE ABOUT PROJECTION??????
In comparison to the vignettes tool explicitly designed to measure hostile attributions, both structured questionnaires define hostility either as a personality style (PSSI) or a component of the aggressive behavior (AQ). Therefore, results obtained through the questionnaires differed from those obtained through the semi-protective tool because changes in the assessment of hostility vary depending on the operational definition of the construct. Since PSSI and AQ measure time-stable personal characteristics (traits), the group mean values didn't differ statistically significant at T1 and T2 and, thus, the questionnaires didn't show useful at measuring aggregate change. Although, prima facie, these results are disappointing it's important to consider the sample size limitations that might have biased the final results leading to false conclusions. As the sensitivity to change is directly affected by the sample size, changes between T1 and T2 might reach statistical significance, even without being clinically meaningful, due to a large sample size. Vice versa, it is possible to obtain a non-significant test because of small sample characteristics rather than lack of sensitivity to change, which is very likely the case in our study. This assumption is supported by the fact that at the individual level the questionnaires showed sensitivity to both positive and negative dynamics, and thus, captured meaningful individual changes taking place over the course of treatment that were overlooked at the aggregate level. Therefore, even though PSSI and AQ were theoretically not expected to be sensitive to change due to the undynamic trait-based operationalization of hostility, they proved variable enough to detect meaningful individual changes. We consider this evidence very valuable. Further, since PSSI and AQ showed sensitive to positive (reduction in HAB) and negative (intensification in HAB) changes, we suppose that the lack of significant results at the aggregate level might be due to the neutralization of the change index because of the similar proportion of positive and negative change.
Nevertheless, statistically speaking both self-report inventories have undeniable strengths. First, for AQ we obtained an internal consistency (Cronbach's ? = .75), identical to the one found in the validation study (Cronbach's ? = .75, Herzberg, 2003). For PSSI, we found an internal consistency (Cronbach's ? = .74), almost identical with the results of the original validation study (Cronbach's ? = .79, Kuhl ; Kazén, 2009). Replicating the results from the original validation studies suggest that the self-report inventories are generally reliable and valid. In addition, both inventories have a broad scaled answering format offering a wide range of response options (a Likert scale range from 1 to 5 for AQ and 0 to 3 for PSSI) allowing participants to state how strongly they identify with particular statements. A detailed Likert scale is a key requirement for assessment tools applied to measure changes. Therefore, changes found at the individual level must be attributed to real change processes that have taken place over the course of treatment rather to the influence of other external factors of contaminating variables. These conclusions suggest that although PSSI and AQ were not initially developed to measure change they possess the potential to determine the dynamic of positive and negative changes in the individual case. Individual changes are of essential importance for both clinical practice and research and have numerous implications for various topics such as treatment progress, responsiveness to interventions and further treatment necessity (Wyrwich ; Wolinsky, 2000; Hays et al., 2005).
Conclusions, implications for clinical practice and recommendations for future research
In order to explore the phenomenon of hostile attribution bias from different perspectives, as well as its change in a course of treatment, we created a complex research question, focusing on multi-method assessment approach. We supposed that the complex nature of cognitive biases can be defined in various ways, e.g. as immanent personality disposition, as social-cognitive interpretation bias, and even as perceptual bias. Since in clinical practice such complex constructs require real-life contextual understanding, the use of a multi-method research perspective is highly relevant for capturing as many elements as possible.
Consistent with our hypothesis, semi-projective methods proved sensitive to identifying the largest proportion of change at both aggregate and individual level and, consequently, seemed to be most informative when it comes to evaluating treatment progress. This is important since most of the classical evaluation assessment instruments are developed to monitor treatment outcomes at the aggregate level for quality improvement and accountability purposes. In contrast, vignettes seemed able to detect intervention change and provide interpretable outcome data at the individual level. Therefore, the use of the vignettes method (and semi-projective tools in general) create opportunities for clinicians to improve individual care for individuals, utilize individual outcome data to guide future treatment, and engage consumers in the treatment process. Nevertheless, clinicians should be cautious while interpreting results obtained with semi-projective techniques and remain aware of the various problems arising from their use.
To summarize, we provide here first evidence for sensitivity to change of different assessment methods. Further investigations of sensitivity to change and/or treatment evaluation should focus on extending the experimental design by adding a comparison group and better controlling of potential interfering variables. Since different statistical methods for identifying change lead to different outcomes, another relevant issue would be to determine which of those methods provide an estimate of change that is most concordant with other distorted cognitions outcomes measures.
We believe that multi-method studies in general represent a promising avenue to broaden the understanding of social-cognitive functioning of the individuals as well to provide the opportunity for developing of effective interventions and evaluation of the treatment efficacy.
Looked at in another way, no instrument clearly outperformed the others in terms of effect sizes across a range of dimensions.
For the perception task:
There are also several possible factors underlying lack of change sensitivity. First, measures may include items that are not relevant to the group under investigation, resulting in lack of measured change (Fitzpatrick et al., 1992). Second, measures that contain items that are categorically arranged (e.g., yes–no or true–false) or that offer only a limited range of response options (e.g., a Likert scale range from 0 to 2) may be scaled in units that are too gross to detect change (Lipsey, 1990). Third, measures may contain instructions not conducive to the detection of change. For example, an instrument that asks respondents to answer items according to how they have felt over an extended period of time (e.g., the past 3 months) is not likely to be useful in detecting changes resulting from interventions that have been delivered weekly over a brief period of time (Berrett, 1998). Fourth, measures may contain items that assess areas that are relatively stable or not a feasible target of assessment (e.g., assessment of state vs. trait constructs). Fifth, measures may contain items that are susceptible to floor or ceiling effects and, therefore, limit the ability of the item to detect meaningful growth or deterioration (Lipsey, 1990).
CHAPTER – I INTRODUCTION Stress is a common problem that affects almost all of us at some point in our lives
CHAPTER ONE 1
Development and usability testing of a self-management intervention to support individuals with obstructive sleep apnea in accommodating to CPAP treatment Suzanne S
Review of Literature Metacognition
Speaking Strategies Use of Students at Leka Nekemte Preparatory School
Faculty of Engineering and Architectural Science Department of Civil Engineering Evaluation Methods for Transportation Projects By Paria SarsharDirected Studies Ryerson University Toronto
Previous PostFocusing on the following pages 21 – 24 starting from Abigail's line " Gah I'd almost forgot how
Next PostNowadays