Hateful Emotional Responses Scale (HatERS): Development and Initial Evaluation

The present investigation consists of two studies designed to provide a preliminary examination of a new scale for measuring hateful emotional responses to being hurt by an offender. Study 1 involved definition of hateful emotional responses, item development, analysis of internal consistency, and convergent and incremental validity. The scale was evaluated in a sample of 33 young (M = 26) college and community adults. A five-item hateful emotional responses scale was developed and psychometric properties of the scale were encouraging. Study 2 involved using the five-item scale developed and evaluated in Study 1 and examining factorial validity and sensitivity to change in hate as a result of attending a lecture on hate reduction. Results of confirmatory factor analysis showed that the scale had acceptable unidimensional factor structure and standardized loadings were all statistically significant and moderate to large in magnitude. The scale also showed expected sensitivity to change. Study 2 participants were 47 middle-aged (M = 50) churchgoers. These initial results suggest that the scale may hold promise for further development in samples of broader, larger, and more varied individuals, communities, and nations.

hate (or love for that matter) an unknown stranger. In this sense, interpersonal hatred comes about as a result of a relationship that has somehow brought about harm. Hatred experienced in this context can have irrevocable damaging effects in terms of satisfaction, intimacy, and love (Aumer, 2016). Conversely, at the intergroup level hatred is often felt for unknown individuals (Fischer et al., 2018). Individuals are hated not for who they are personally or what they may have done to another individual, but rather hated groups are members of an outgroup that are hated for what they represent (Fischer et al., 2018). In the case of interpersonal hatred the emotional response is toward a specific person with an offending action or character, but in the case of intergroup hatred the emotional response might be fueled by in-group indoctrination, collective victimhood that promotes persistent narratives of hate, and distancing between the in-group and hated outgroup (Fischer et al., 2018).
Intergroup hate has received significant attention from public commentators and scholars alike, and for good reason. Intergroup hatred is daily on display in politics and society, as well as, in age old disputes between cultures, religions, and nations. Fischer et al. (2018) suggested that hate may be the most group-based emotion and that interpersonal hate can quickly explode into intergroup hate fueling political and societal conflict. Nevertheless, understanding intergroup hatred alone offers only a partial understanding of the construct of hate. Interpersonal hatred is a common experience and salient for many. Indeed, Aumer, Bahn, and Harris (2015) suggested that people rarely mention acquaintances and groups of people as targets of hate, rather specific offenders in existing spousal, family, or friend relationships are more common targets. The nature of hate being something commonly experienced as an emotion at both interpersonal and intergroup levels requires that both levels of the construct be studied. That said, the study of interpersonal hate appears to be lagging behind intergroup hate and measurement tools that facilitate the study of both could offer advantages such as easy comparison across interpersonal and intergroup contexts.
Given the predominance of emotional expression in the definition and discussion of hate, the present study focused on developing a measure of hateful emotional responses. Our definition falls close to that of others as we defined hate as an intense, emotional response experienced by an individual following perceived wrongdoing or perceived ill-will on the part of another person or group of people (Allport, 1954;Brudholm, 2010;Fischer et al., 2018). Based on this definition we developed a scale to assess hateful emotional responses.
Our scale development had as its first priority to focus keenly on hate as an emotion. Scales that measure hate are not common and those that do exist can be multifaceted and complex (Sternberg & Sternberg, 2008) and in some cases confounded with constructs such as amends-making, apology, and avoidance. Take, for instance, one of the most common measures of hate, the Triangular Hate Scale. This scale contains three dimensions of hate that map onto the Triangular Theory of Hate (Sternberg & Sternberg, 2008). These include: (a) negation of intimacy, (b) passion, and (c) commitment. The scale has been used in research specifically focused on hate and is useful for that purpose. However, the scale is meant to be used in its entirety. The theory, along with the scale, does not make much sense if one component of the model is measured in isolation as the three components are used to create seven different types of hate (e.g., cool, hot, cold) depending on the scores on the three subscales. The complexity of the theory and scale make its use viable in specific studies of the Triangular Theory of Hate, but for researchers, political scientists, or pollsters interested in simply gauging hateful emotions it might have considerably less utility. A shorter adaptation of the Triangular Hate Scale has recently been developed which includes 12 items, but it has not been examined to establish its reliability and validity (Zeki & Romaya, 2008). A third hate scale developed by Aumer et al. (2015) provides another example of an existing measurement tool for assessing hate. This measure however also requires several considerations. Although it is specifically focused on interpersonal hate, it contains 27 items that appear to assess a mix of emotions and motivations with several items that may be confounded with other measures (e.g., "harm," "pain," "distress," "apology," and "amends"). The measure, while used in studies of interpersonal hate, has not been validated. Therefore, we chose to focus exclusively on developing a measure of hateful emotion-what we believe to be and define as the key aspect of hate.
A second priority in the development of the present hate measure was to make it useful for a broad number of professionals in a wide array of disciplines. Hateful rhetoric, political actions, and crimes (FBI, 2017) have been and continue to be high, as is interest in understanding hate and addressing its consequences (Delgado et al., 2017;Green, McFalls, & Smith, 2001). For instance, major polling firms are seeking to track and understand hate in America (Harris Poll, 2015;Smith et al., 2019). Pollsters and public opinion surveys typically require exceptionally brief (rarely more than 4-5 items) and flexible measurement tools capable of being used in a variety of contexts. Regrettably, the requirements for brevity in these circumstances often mean that single-item assessments are used for which it is difficult to establish reliability and validity. Sometimes two or three item assessments are used, but these commonly possess unacceptable levels of measurement error and are rarely evaluated for construct validity meaning that these are crude measurement instruments at best. Often these scales, while brief, at most offer face validity. For these reasons, we aimed to develop a measure that would meet both brevity and validity requirements. Furthermore, the scale under development, with but a few minor adjustments, could be adapted for use with a variety of individuals, groups, or entire cultures facilitating comparisons of interpersonal and intergroup hateful emotional responses using the same measurement tool.

Present Study
Two studies were conducted to develop and evaluate the Hateful Emotional Responses Scale (HatERS). Study 1 involved item generation, refinement, initial evaluation of item-scale statistics, estimated internal consistency, and convergent and incremental validity analyses in a small sample of college students. A second study evaluated the unidimensionality and internal consistency of the HatERS scale, as well as, its sensitivity to change in response to a lecture on hate in a group of churchgoers.

Study 1
The purpose of Study 1 was to develop items focused on hateful emotions in response to a wrongdoer's actions. We examined the internal consistency of these items, their convergence with another hate scale, a forgiveness scale, and single-item hate indicator, and the incremental validity of the HatERS scale.

Method
Participants and procedure Study 1 participants consisted of 33 college undergraduates and community members. This was a convenience sample obtained through snowball sampling (research team members recruited from their own contact lists and asked willing participants to pass the survey along to others) and the only inclusion criteria was English literacy. Participants were 26 years old on average (Mdn = 21; SD = 10.86) and 70% were female. Participants received an email with a link to an online questionnaire. Upon opening the questionnaire the first page contained an informed consent form with options to consent or not consent and in the case of the latter individuals were brought to the end of the survey where they were thanked for their time. Study 1 was approved by the institutional review board at Luther College.

Measures
The HatERS scale was developed by the Laboratory for the Investigation of Mind, Body, and Spirit research team led by Dr. Loren Toussaint at Luther College. Seven team members (two doctoral-level researchers and five students) were assigned published literature to read and discuss on the conceptualization and definition of hate and were asked to write several items that would clearly reflect the team's agreed upon definition of hate. Doctoral-level team leaders wrote items themselves and reviewed, revised, and/or rewrote scale items from each other and from students in a collaborative fashion. Items were written to be responded to on a Likert-type response scale of 1 (strongly disagree) to 7 (strongly agree). The initial collection of items totaled about 30 and based on the fit with our definition of hate, item clarity and wording, and general fit with other items, items were selected in or eliminated from consideration. An iterative process of writing, discussion, and re-writing, ultimately resulted in five items which singularly focused on the dimension of emotional reaction to wrongdoing, as the agreed upon definition stated, and were clearly written, concise, and had good team consensus regarding fit with the overall construct of interest (see Appendix A). Scoring the HatERS was intended to be straightforward to encourage its use by individuals who may not have a strong background in measurement and requires only an average (or sum) of the responses to the five items. No reverse scoring or item weighting is required. In addition, three other measures were included for the purposes of evaluating convergent validity. One measure assessed hate , and included 27 items assessing multiple aspects of hate including emotions, motivations, and attitudes. This measure was chosen because it reflects an interpersonal orientation to hatred-responding to a wrongdoer-that is similar to the present measure under development. Each item was responded to on a 1 (very slight or not at all) to 5 (extremely) response scale. A second measure was a single-item, face-valid indicator of hate that simply read, "I hate this person." This item was responded to on a Likerttype response scale of 1 (strongly disagree) to 7 (strongly agree). The third measure was a measure of forgiveness of a specific person for a specific offense (Brown & Phillips, 2005), and included seven items. Each item was responded to on a 1 (strongly disagree) to 7 (strongly agree) response scale. Both multi-item convergent validity measures possessed adequate internal consistency, .89 and .84, respectively for the hate and forgiveness scales.

Analyses
Analyses for Study 1 included item-scale statistics, coefficient alpha, Pearson correlations, descriptive statistics (mean and standard deviation), and hierarchical regression. No violations of statistical assumptions were observed and statistical significance was set at p < .05.

Results
The five-item HatERS scale showed good internal consistency (α = .82), especially given its brief nature. Item-total correlations ranged from .47 to .74, also confirming the internal consistency of the scale. The HatERS scale also showed expected convergence with the existing hate scale (r = .68, p < .001), single-item measure of hate (r = .77, p < .001) and with forgiveness (r = -.78, p < .001). To evaluate incremental validity a hierarchical regression equation was specified where forgiveness was the outcome variable and the hate scales were the predictors. The other two hate scales were entered on step one (R 2 = .62, p < .001) and the HatERS scale was entered on step two (R 2 = .75, p < .001). The HatERS scale showed incremental validity in predicting an additional 13% of the variability in forgiveness above that already accounted for by the other two hate scales (see Table 1). After averaging the responses for the five items for each individual, the group average on the HatERS scale was computed and found to be 4.12 and the standard deviation was 1.29. The observed range of score was 1.2 to 6.8 and the possible range of scores is 1 to 7.

Study 1 aimed to develop an initial scale for assessing hateful emotional responses to being hurt by another individual.
A team-based approach to scale development was used where team members became familiar through reading and discussion with important literature on hate, derived an agreed upon definition of hate, wrote items that mapped onto that definition, discussed and revised items that were written, and ultimately selected five items for inclusion on the HatERS scale. In the scale's initial fielding with college students and community members promising results were obtained suggesting an internally consistent scale that shows expected construct validity with other hate scales and a forgiveness scale, and incremental validity above the other hate scales. The scale is sensitive to individual variations in hate showing a wide range of observed scores. While this is encouraging, Study 1 does not offer insight into factorial construct validity and hence no strong evidence of unidimensionality. Furthermore sensitivity to change (i.e., construct validity) in hate is not demonstrated in Study 1. Hateful emotional responses are expected to change across time and situations so the measure should show an ability to detect differences in emotional experiences of hate across time in the same individuals. Study 2 was designed to address these issues.

Study 2
The purpose of Study 2 was to examine the factorial validity of the HatERS and evaluate its sensitivity to change in hate levels in response to a lecture on hate that encouraged congregants to moderate their hateful emotions.

Participants and procedure
Study 2 participants were 47 churchgoers attending a lecture on hate. Participants were 50 years old on average (Mdn = 55; SD = 21.63) and 53% were female. Participants were recruited through advertisement in the church bulletin and verbal invitation after attending a Sunday church service. The only inclusion criteria was English literacy. Before the lecture an informed consent statement was given to potential participants that was read, signed, and returned. Participants then received a paper version of the HatERS scale that was completed before and after the lecture. Study 2 was approved by the institutional review board at Luther College.
The lecture on hate (approximately 20 minutes in length) was designed to achieve three main objectives. First, consistent with our definition, the lecture aimed to define hate as primarily an emotion. Second, the intensely negative and potentially harmful effects of hate were discussed. In this part of the lecture hate was contrasted with other constructs such as anger, love, and forgiveness. Third, attendees were asked to consider that negative emotions (e.g., anger) following a wrongdoing are natural and to be expected. But, they were also asked to consider that hateful emotional responses could be destructive and are to be avoided when possible. Attendees were admonished to control hateful emotions to prevent unnecessary and negative interpersonal or social ramifications. The lecture ended with summary points, questions, and discussion lasting another 10 minutes. The overarching goal of the lecture was to urge attendees to recognize the destructive nature of hate and control hateful emotional responses they might experience. Simply put, the message was to reduce hate. The only measure included in Study 2 was the HatERS scale. The scale was used exactly as it was in Study 1, with the one exception being that it was administered in paper-and-pencil format.

Analyses
Confirmatory factor analysis was used to evaluate the unidimensionality of the HatERS scale. Chi-square and CFI were used as fit indices and standardized loadings are reported. The commonly reported fit index RMSEA was not used as it is not recommended for models, such as the present one, with few degrees of freedom and small sample size (Kenny, Kaniskan, & McCoach, 2015). Paired t-tests were used to evaluate pre-post changes in hate.

Results
A confirmatory factor analysis model was specified to test the unidimensional structure of the HatERS scale at pretest prior to the hate lecture. The model fit the data acceptably well, χ 2 = 11.02, p = .05; CFI = .95. Standardized factor loadings were all statistically significant (ps < .001) and ranged from .55 to .92. Inspection of modification indices revealed that allowing the error terms of two items, that specifically named anger and resentment, to correlate modestly improved model fit, χ 2 = 6.23, p = .18; CFI = .98. Standardized factor loadings were virtually unchanged, ranging from .51 to .93, and all were again statistically significant (ps < .001) (see Table 2). The scale again showed good internal consistency (α = .87). A paired t-test was used to evaluate change in hate from pretest to posttest. Prior to the lecture on hate, the mean level on the HatERS was 3.54 (SD = 1.43) and following the lecture on hate the mean level on the HatERS was 3.23 (SD = 1.47). This difference was statistically significant and small in size, t(43) = 2.54, p < .05; d = . 22.

Discussion
Study 2 demonstrated that the HatERS was unidimensional in structure and possessed sensitivity to change. Fit indices of both confirmatory models were acceptable and factor loadings indicated that all items were strongly connected to the construct. The t-test results indicated that even with a short-term and modest "intervention" to reduce hate, the HatERS scale was capable of detecting the change across time within the same individuals.

General Discussion
The present studies offer preliminary evidence in favor of the HatERS. The scale is brief, consisting of only five items, and focuses on the emotional response of hate when an individual is wronged. Findings from this work suggest that the scale is unidimensional, internally consistent, and sensitive to change. Further, the scale showed strong convergence with hate and forgiveness scales. The scale also provides incremental information above other hate measures.
The common interest of philosophers, theologians, and social scientists in the emotional experience of hate (Ahmed, 2001;Brudholm, 2010;Rempel & Burris, 2005;Solomon, 1993;Sternberg, 2003;Sternberg & Sternberg, 2008;VandenBos, 2007) may be well served by using the HatERS in empirical research. As just a few examples consider that religious scholars may be interested in understanding the experience of hateful emotions across religious affiliation or across conflicted religious groups. Sociologists may find it interesting to assess hateful emotions across social strata such as race, education, or income. Scientists in public health may be eager to know the effects of hateful emotions on health outcomes, especially after large-scale hate crimes and other socio-political conflicts and confrontations. In each case, addressing these research questions takes as a given that reliable and valid measurement tools are available. We believe that the HatERS offers one such option and can therefore serve the needs of many scholarly and applied socio-political researchers. Perhaps more so than existing measures, the HatERS offers a simple and efficient means for including an assessment of hate that is not overly complicated or tied to only one specific perspective or framework, thereby offering flexibility in conceptualization and implementation. Note: All loadings significant at p < .001.

Limitations and Future Directions
Although the present studies can provide preliminary evidence of the utility of the scale, future work could further expand the usefulness of a scale like this by addressing some of the study limitations. For instance, a limitation of the current studies is the small and specific nature of the samples. Having said that, the study included not only college students but community members and churchgoers. Nevertheless, larger and more socio-demographically and geographically diverse samples are needed for continued use, validation, and refinement. Another limitation of the current study is the limited nomological network that the current measure was validated within. The scale showed expected convergence with other measures of hate and forgiveness. It also showed incremental and factorial validity adding to its construct validity. However, additional convergence and divergence measures could be used to strengthen construct validity including such things as anger, empathy, vengeance, and love, among others. The present studies provide only a beginning in the way of mapping the nomological network in which hateful emotional responses are embedded. Future studies should continue to examine the psychometrics of the scale in multiple contexts. The present usage of the scale was confined to interpersonal types of hate, but minor adjustments to the wording of the instructions and items could allow the scale to be easily used in group, community, and national or international settings (see Appendix B). Polls and government surveys may find the scale useful in other ways and will likely find the brevity, unidimensionality, and ease of scoring attractive. It is important to note, however, that while the psychometric properties and ease of use may make the HatERS an attractive option, it was not designed with the intent to be used in individual counseling. For use in those contexts increased internal consistency and predictive validity would be useful. This notwithstanding the measure could be evaluated and refined for use in clinical and counseling settings. The HatERS has preliminary support and the initial results are encouraging and our hope is that others will find the scale useful and implement hate assessment in new and insightful ways. Only with continued development and refinement of measurement tools can we expect the science of hate to progress.