Story ID
Part 3: Analysis by Rater Categories
subjects’ mood may affect the emotional content of the narratives. The Positive and Negative Affect Schedule measures the general mood of a participant using two orthogonal scales: Positive Affect (PA) and Negative Affect (NA)229. Together, these scales provide a general picture of a participant’s mood. Comparison of PA and NA scores for the subjects who told the narratives with the greatest percentage of emotional words with those for the subjects whose narratives contained no emotional words showed no significant difference for either positive emotional words (for PA, t[117] = 0.14, p = 0.89; for NA, t[117] = 0.42, p = 0.68) or for negative emotional words (for PA, t[168] = 0.36, p = 0.72; for NA, t[168] = 1.81, p = 0.07).
raters to read each narrative in the context intended. Graphing the resulting data also makes it easy to get a sense of the overall contents of the database.
Figure 36 shows the actions that appear in the narratives. Most prominently, there are 533 ratings about narratives where the narrator lied and 507 ratings where the
participant was helping someone. The raters were asked to use a scale from 1-3 when categorizing the ratings to designate how well the narrative exemplified the category.
They were also asked to note stories in which the participant specifically chose to do the opposite of the category. For example, if the narrator decided not to cheat on a business deal, the rater would give the category ‘cheating in business’ an opposite rating of 1-3.
This allows future users of the database to more easily select narratives where the narrator chose not to make a bad moral decision230. The rating categories are shown as a stacked bar graph (Figure 36 - Figure 38) to display the contribution of each to the total number of ratings in that category. The reasons narrators most commonly gave for their actions can be seen in Figure 37. They were motivated by selfishness (n = 702 ratings) and doing the wrong (n = 646) and right thing (n = 596). Finally, raters were asked to categorize how the narrators felt about their actions (shown in Figure 38), where we find that subjects mostly felt regret about their behavior (n = 893) and doing the wrong thing (n = 835). These emotions are not surprising since all of the narratives concern morally laden events. Along with categorizing the feelings of each narrator, the raters were asked to rate how much rationalization was given for the narrator’s actions. In 1565 of the 2169 ratings (72.2%), the narrators provided some sort of justification for their behavior.
In designing the rating sheet, a subsample of 100 narratives was used to generate the categories included. During that process, the hypothesis arose that subjects might
230 It also allows users to choose narratives in which the subject specifically chose to do the wrong thing.
change their moral behavior based on the location of their actions231. Generally, it seemed that subjects had different moral standards for different situations (work versus home, situations where children or animals were involved, etc.). In addition, knowing about where the narratives took place (as well as salient characters in those locations) provides another way to ‘see’ into the entire database at once. A group of 8 ‘scenes of action’ were developed from the subsample and were included on the rating sheet. Table 10 displays the results of those ratings. Only 11.6% (n = 251) of the ratings had no scene chosen while 37.5% (n = 831) of the ratings found that the narrative involved a family member or close friend. Not surprisingly, 29.2% of ratings also found that the narrative took place at home or in the narrator’s neighborhood. Raters were encouraged to write in other scenes that seemed important to the narrative. These scenes are enlightening in
themselves, providing insight about other places where important moral decisions occur including places such as jail, hospital, rehab and on the internet.
After categorizing each narrative, raters were asked to give their own opinion232 of how right or wrong the action in the narrative was (using a scale from -5 (wrong) to +5 (right)). Raters were told to use ‘0’ to represent a morally ‘gray area’ and to leave the rating blank or note if they felt that the narrative did not involve a moral situation. Only 3.3% of the ratings (n = 72) were thought by some raters not to be moral233. Figure 39 shows two different representations of the distributions of the raters’ judgments. The first
231 One example of this is Memory 5096, a narrative where the subject decides while on vacation abroad to have sex on the beach with her boyfriend even though it is illegal. She regrets her decision in part because she knew she wouldn’t have to live with the consequences but her foreign boyfriend did.
232 As was discussed in Chapter 3, this is NOT an unbiased sample and these data about the morality of the ratings should only be used as a way to separate the data. Any ratings for comparison to other groups should be done with an unbiased sample group.
233 It is important to remember that some memories were chosen for inclusion in the database even though neither researcher reading the memories felt it was moral. This occurred when the subject specifically said that they felt the action was right or wrong. The database criteria were intentionally broad enough to allow the inclusion of these narratives.
(represented by the blue bars) is a distribution that shows the number of ratings that received the score shown on the x-axis. Thus, 130 ratings (5.9%) received a score of -5 from any rater. The red bars represent the number of narratives with a mean score as shown on the x-axis. There are 7 narratives that received a mean score of -5. This means that every rater believed that these narratives234 were as morally wrong as the scale allowed. Together these distributions show us that individual raters rate similarly to the group, although the extremes (-5 and +5) are not used quite as reliably, which is to be expected with Likert scale ratings.
Any generalized description of these 758 moral narratives is complicated by the individuality and complexity of each narrative (as well as the number of narratives).
However, using the categorizations generated by the raters provides an excellent overview of the contents and breadth of the narratives.