PUNISHMENT AND THE POTENTIAL FOR NEGATIVE REINFORCEMENT WITH HISTAMINE INJECTION
PAULOCÉSARMORALESMAYER1, MARCUSBENTES DECARVALHONETO2,AND JONATHANL. KATZ3
1UNIVERSIDADE ESTADUAL DO OESTE DO PARANÁ - UNIOESTE
2UNIVERSIDADE FEDERAL DO PARÁ - UFPA
3NATIONAL INSTITUTE ON DRUG ABUSE–INTRAMURAL RESEARCH PROGRAM
The present study examined punishment of responding with histamine injection, and its potential to generate avoidance of punishment. Sprague–Dawley rats were trained under concurrent schedules in
which responses on one lever (the punishment lever) produced food under a variable-interval schedule, and under some conditions intermittent injections of histamine, which suppressed behavior. Responses on a second (avoidance) lever prevented histamine injections scheduled on the punishment lever. After stabilization of punished responding, a variable-interval 15-s schedule of cancellation of histamine (avoidance) was added for responding on the second/avoidance lever, without subsequent acquisition of responding on that lever. Progressive decreases in the length of the punishment variable-interval schedule increased suppression on the punishment lever without increases in response rates on the avoidance lever. Exchanging contingencies on the levers ensured that response rates on the avoidance lever were sufficiently high to decrease the histamine injection frequency; nonetheless response rates on the avoidance lever decreased over subsequent sessions. Under no condition was responding main-tained on the avoidance lever despite continued punishing effectiveness of histamine throughout. The present results suggest that avoidance conditioning is not a necessary condition for effective punish-ment, and confirm the importance of empirical rather than presumed categorization of behavioral effects of stimulus events.
Key words:punishment, suppression, avoidance, negative reinforcement, histamine, lever press, rats
Most experiments involving punishment use electric shock as the punishing stimulus, which has well-documented advantages such as its quantifiable physical dimensions (shock inten-sity, period, and duration). Those dimensions of electric shock are readily controlled, allowing its use across a wide range of values, and
facilitating replication and direct comparisons across studies (Azrin & Holz, 1966; Barker et al., 2010; Dinsmoor, 1998). Among previously noted drawbacks of electric shock are its well-documented eliciting functions (e.g., Flaherty, 1985), which may contribute to competition between responses, and may complicate the interpretation of its effects.
The generality of the behavioral principles established with electric-shock punishment can be tested or extended by using novel stim-uli (Barker et al., 2010; Branch, Nicholson, & Dworkin, 1977; Catania, 2013; Church, 1969; Dinsmoor, 1998). Although infrequently employed, examples of various punishing stim-uli include: intense noise (e.g., Friedel, DeHart, & Odum, 2017; Holz & Azrin, 1962), air blasts (e.g., Carvalho Neto et al., 2005; Spealman, 1978), bright light (e.g., Barker et al., 2010), species-specific stimuli (e.g., Mas-serman & Pechtel, 1953), and time out from positive reinforcement (e.g., Ferster, 1958; Herrnstein, 1955; though see Leitenberg, 1965, for the varied effects of time out). Each of these stimulus groups have their own strengths and weaknesses, the latter often The authors thank Drs. Kennon A. Lattal, M. Jackson
Marr, Frans van Haaren, and John V. Keller for comments on a previous version of the manuscript. All experiments were approved by the NIDA IRP Animal Care and Use Committee.
The data reported herein constituted part of the requirements of Paulo César Morales Mayer for the degree of Doctoral in Behavioral Theory and Research at Universidade Federal do Pará - UFPA, with Marcus Bentes de Carvalho Neto as mentor (CNPq Grant #311603/2016-5). The experiments were conducted at the Intramural Research Program of the National Institute on Drug Abuse. Dr. Mayer’s time at NIDA-IRP was funded in part
by scholarship from the CAPES (PDSE#3878/13-9), Minis-try of Education, Brazil.
Address correspondence to Jonathan L. Katz, NIDA Intramural Research Program–Molecular
Neuropsychia-try Research Branch, 251 Bayview Blvd., Baltimore, MD 21224. Phone 001-443-812-1415; e-mail JKATZ@intra.nida. nih.gov.
doi: 10.1002/jeab.319
including, but not limited to, a lack of precise physical dimensions, habituation, and unscheduled avoidance.
Consequent histamine injection is an alter-native punishing stimulus that has been stud-ied with some frequency and with several of the advantages of an ideal punishing stimulus, as described by Azrin and Holz (1966). For example, it can be used across a wide range of magnitudes (see dose-effect curves in Gold-berg, 1980; Katz & GoldGold-berg, 1986; Podle-snik & Jimenez-Gomez, 2013; Sharpless, 1961). Further, histamine has precise physical/phar-macological specifications; the actual contact with the organism is constant from one occur-rence to the next (there are no indications of pharmacological chronic tolerance or tachy-phylaxis), and options for escape or avoidance are not possible unless specifically pro-grammed (e.g., Takada, Winger, Cook, Larsc-heid, & Woods, 1986). Histamine has been used as an alternative to electric shock as a punisher in various schedule arrangements (e.g., Goldberg, 1980; Holtz & Carroll, 2015; Negus, 2005; Podlesnik, Jimenez-Gomez, & Woods, 2010). Though other drugs have also been used (Koffarnus & Winger, 2015; Prada, Takada, Katz, Goldberg & Barrett, 1987; Takada, Barrett, Allen, Cook, & Katz, 1992), histamine may be best suited among drugs for use as a punisher due to its fast onset and off-set (Goldberg, 1980). Finally, histamine, at the doses presently utilized, elicits no known skele-tal responses (Kuraishi, Nagasawa, Hayashi, & Satoh, 1995; White & Rumbold, 1988), and the known elicited cardiovascular effects are pharmacologically independent of its punish-ing effects (Goldberg, 1980; Podlesnik & Jimenez-Gomez, 2013).
The generality of the behavioral principles established with electric shock have been extended with the use of histamine. Studies have established that as with electric shock, suppression of behavior is related to the mag-nitude of the stimulus (e.g., histamine dose: Goldberg, 1980; Katz & Goldberg, 1986; Podle-snik et al., 2010), injection delay (Woolverton, Freeman, Myerson, & Green, 2012), and vari-ous scheduling conditions of the procedures employed (e.g. Negus, 2005; Podlesnik & Jimenez-Gomez, 2013). Additionally, the effects of drugs on punished responding with histamine appear to be largely similar to those with electric-shock punishment (e.g. Goldberg,
1980; Katz & Goldberg, 1986). Finally, as with electric shock, histamine has also been shown to function as a negative reinforcer (e.g. Takada et al., 1986).
Azrin, Hake, Holz and Hutchinson (1965) studied punishment and concurrently, the potential of that punishment to generate nega-tive reinforcement of another response. Pigeons were trained to peck one key with food reinforcement under a fixed-ratio (FR) 25-response schedule. After electric-shock punishment was introduced as a conse-quence of each response, another contingency was introduced in which a response produced a change in overall illumination in the cham-ber and eliminated punishment for the remainder of the fixed ratio. As expected, punished response frequency was an inverse
function of punishment magnitude (shock intensity), whereas the escape response fre-quency was a direct function of punishment magnitude. These relations were obtained with different schedules of food reinforce-ment, and the escape response showed schedule-appropriate patterns when rein-forced under fixed-interval and fixed-ratio schedules. In a latter discussion of that fi nd-ing, Azrin and Holz (1966) stated that “a
major effect of punishing a response is to gen-erate a strong tendency on the part of the sub-ject to escape from the punishing situation entirely”(p. 408).
Lattal and Cooper (1969) also trained pigeons to peck a key under an FR 25 sched-ule of food presentation and punished this responding on an FR1 schedule of electric-shock presentation. The punishment contin-gency could be avoided if any two responses were interspaced by a specified interval (15, 30, or 45 s), a contingency which resourcefully exploits its compatibility with operant response suppression. As in the study by Azrin et al. (1965), meeting the response omission requirement produced a change in the color of the response key allowing the completion of the FR 25 sched-ule of food presentation without punish-ment. Avoidance responding was maintained most prominently at the lower omission time requirements.
schedule of electric-shock presentation was added to the schedule of food presentation which resulted in a small punishing effect. Once performances stabilized, an avoidance contingency was added such that the electric shock could be avoided if the time between two successive responses was greater than 5, 10 or 30 s (during different phases of the study). In contrast to the previous studies, meeting the avoidance contingency produced no exteroceptive stimulus change. With the introduction of the avoidance contingency the shock frequency initially decreased six-fold or greater. Further, the response rate was an inverse function of the avoidance interval, as would be expected for avoidance responding (e.g. Sidman, 1953). Those results suggested that a punishment contingency can be a suffi -cient predisposing condition for the establish-ment of avoidance responding.
The present series of experiments extended those by Arbuckle and Lattal (1987) further investigating the relation between avoidance and punishment with histamine injection as the punishing stimulus. In the Arbuckle and Lattal study, the avoidance response was a par-ticular spacing of responses on the same oper-andum as the punished response. As the duration of interresponse times among pun-ished responses can be selectively modified by punishment contingencies alone (Everly & Perone, 2012; Galbicka & Branch, 1981), the present study extended the analysis to assess the effects of punishment and avoidance with contingencies arranged on different levers (topographical tagging, Catania, 1973).
Method
Subjects
Seven male Sprague–Dawley rats (Taconic
Farms, Germantown, NY), weighing approxi-mately 300 g at the start of the study served as subjects. One subject (PM06) died prema-turely and was replaced with another (PM16).
The subjects were acclimated to a
temperature- and humidity-controlled vivar-ium for at least 1 week with a 12-hr light/dark cycle (lights on at 07:00 a.m.) during which food (Scored Bacon Lover Treats; Bio-Serv, Frenchtown, NJ) and tap water were available at all times. After acclimation, body weights were maintained at approximately 325 g by adjusting daily food rations. Animals had free
access to water at all times in the home cages. Care of the subjects was in accordance with the guidelines of the National Institutes of Health and the National Institute on Drug Abuse, Intramural Research Program, Animal Care and Use Program, which is fully accre-dited by Association for Assessment and Accreditation of Laboratory Animal Care International.
Surgical preparation and postsurgical care. Under anesthesia (ketamine 60.0 mg/kg and xylazine 12.0 mg/kg, i.p.) chronic indwelling catheters were surgically implanted in the right external jugular vein. Catheters exited at the midscapular region of the subject through a back mount which allowed connection with the subject to the injection pump. After sur-gery subjects received heparinized saline (50 IU/ml, i.v.) and subcutaneous antibiotic (enrofloxacin, 5.0 mg/kg in saline) daily to minimize the likelihood of infection and the formation of clots or fibroids. Catheter patency was checked about every 10 days with methohexital injection (1.5 mg/kg). If the catheter failed, it was removed and after at least 7 days another vein (either left jugular, right femoral or left femoral) was catheterized. All subjects were allowed to recover from sur-gery for approximately 7 days before returning to behavioral procedures.
Apparatus
Daily sessions were conducted using six operant-conditioning chambers (modified ENV-203; Med Associates, St. Albans, VT) that measured 25.5×32.1×25.0 cm. The
cham-bers were enclosed within sound-attenuating cubicles that were equipped with a fan for ven-tilation and were provided with white noise to mask extraneous sounds. The front wall of each chamber contained a house light (28 V, 100 mA) at the ceiling and two response levers, mounted 5.0 cm from the midline and 4.0 cm above the grid floor. A downward dis-placement of a lever with a force approximat-ing 0.20 N defined a response and always activated a relay mounted behind the front wall of the chamber producing an audible
“feedback” click. Six light-emitting diodes
Associates) was mounted on the midline of the front wall between the two levers and 2.0 cm above thefloor. The three green LEDs above the lever on which the response was emitted flashed on for 0.1 s with each food pellet delivery. An injection pump (Model 22; Harvard Apparatus, Holliston, MA, set at a rate of 4.22 ml/min.) placed above each cubicle delivered injections of specified volumes from a 20-ml syringe. The syringe was connected by Tygon tubing to a single-channel fluid swivel (375 Series Single Channel Swivels; Instech Laboratories, Plymouth Meeting, PA) that was mounted on a balance arm above the cham-ber. Tygon tubing from the swivel to the sub-ject’s catheter completed the connection to
the subject and was protected by a surround-ing metal sprsurround-ing. The three yellow LEDs above the lever on which the response was emitted flashed on for 0.1 s with each histamine or saline injection.
Drugs
Histamine dihydrochloride (Sigma-Aldrich, St. Louis, MO) was dissolved in sterile saline (0.9% NaCl) solution (4.28 mg/ml). Heparin solution was used toflush the catheters, before and after the sessions. Methohexital was administered on occasion to test catheter patency.
Procedure
The house light was turned on at the start of each experimental session. Subjects were initially trained to press both levers with food reinforcement (45-mg food pellets; Bio-Serv) and performances were allowed to stabilize under concurrent variable-interval (VI) sched-ules on the two levers. The VI schedule con-sisted of 20 interval values in a mixed progression (Fleshler & Hoffman, 1962). Sub-sequently, the schedule for the lever on which most responding occurred was changed to extinction (EXT) and the schedule on the alternate lever was set at VI 120-s. When response rates were again stable the venous catheters were implanted and after 7 days, subjects were placed back on the concurrent VI 120-s EXT schedule. Additionally, a VI 15-s schedule of saline injection (consisting of 12 intervals arranged as those for food rein-forcement) was introduced on the lever on
which responses continued to produce food according to the VI 120-s schedule. All sessions lasted 1 hr and were conducted at about the same time daily in the afternoon. This training phase lasted between 23 and 71 sessions (see Table 1).
Histamine punishment. After three sessions on the above schedule, response-produced his-tamine injections were introduced. Under this condition, i.v. histamine injections (0.3 mg/ kg/injection) replaced saline and were deliv-ered according to the VI 15-s schedule (con-joint VI 120-s [food], VI 15-s [histamine]). The dose was selected based on pilot studies and previously published papers (e.g. Podlesnik & Jimenez-Gomez, 2013). This lever is hereafter referred to as thepunishment lever.If both a pel-let and an injection were simultaneously sched-uled for a response, the pellet was delivered following that response and histamine was delivered following the next response. The schedule for responses on the alternate lever remained EXT.
Punishment avoidance. Once performances with histamine injections were stable from one session to the next, a VI schedule of punish-ment avoidance replaced EXT on the alter-nate lever. Under this schedule, a single response on the alternate lever (hereafter referred to as theavoidance lever) canceled the next histamine injection scheduled according to the VI 15-s schedule on the punishment lever. The response on the avoidance lever cancelled only the presentation of the next histamine injection scheduled on the punish-ment lever. Additional responses on the avoid-ance lever had no scheduled consequences until the start of the next interval within the punishment VI schedule. All scheduled hista-mine injections could therefore be avoided if the subject made at least one response on the avoidance lever within every interval of the VI 15-s schedule. This procedure was similar to that used for electric shock (e.g., de Villiers, 1974), and has been referred to as a schedule of deletion as distinct from a schedule of post-ponement (e.g. Perone & Galizio, 1987). The sequence and number of sessions at which the above conditions were executed are shown in Table 1.
increase the degree of potential reduction in histamine injection frequency produced by avoidance responses. It was anticipated that a greater change in histamine injection fre-quency produced by a response would facili-tate acquisition of responding on the avoidance lever (e.g., Herrnstein & Hineline, 1966). Initially sessions with the VI schedule of histamine cancellation alternated with sessions in which responses on the avoidance lever had no effect (EXT; Table 2). The EXT conditions were later eliminated in an attempt to facili-tate acquisition of responding on the avoid-ance lever. As the longer punishment VI values proved ineffective in establishing avoid-ance responding, some subjects were studied
only at the shorter punishment VI-schedule parameters (Table 2).
Once performances stabilized with the VI 1-s 1-schedule of hi1-stamine cancellation, the scheduled contingencies on the two levers were reversed. This change ensured that response rates on the newly designated avoid-ance lever were sufficiently high to actually reduce the frequency of histamine injections. As a result, the relatively high response rate on the avoidance lever decreased the fre-quency of histamine injections. Because results following the reversal in lever contingencies varied among subjects (see Results) reversals were conducted repeatedly with several sub-jects (Table 3). As responding appeared to be Table 1
Sequence of procedures during thefirst histamine punishment condition
Lever 1 Lever 2 Sessions
Schedule 1 Schedule 2
Histamine dose
(mg/kg) Schedule 1 PM01 PM02 PM03 PM04 PM05 PM06 PM16
VI 180 (pellet) - - VI 180
(pellet) 1-45 01-46 1-39 1-41 1-36 1-44 1-13 VI 120 (pellet) - - Extinction 46-71 47-59 40-68 42-67 37-67 45-64 14-23 VI 120 (pellet) VI 15 (inj) 0 Extinction 72-74 60-62 69-71 68-71 68-70 65-67 24-26* VI 120 (pellet) VI 15 (inj) 0.3 Extinction 75-77 63-67 72-75 72-73 71-74 68-72
VI 120 (pellet) VI 15 (inj) 0.3 Avoidance 78-83 68-73 76-81 74-79 75-79 73-78 VI 120 (pellet) VI 15 (inj) 0 Extinction 84-89 74-79 82-86 80-84 80-84 79-83 VI 120 (pellet) VI 15 (inj) 0.3 Extinction 80-86 87-90 85-88 85-89 84-87 VI 120 (pellet) VI 15 (inj) 0 Extinction 87-89
Note.The table displays a description of contingencies on each lever, the number of sessions conducted with each subject on those conditions, and the dose of histamine used. A dose of“0”indicates saline injections.
*For this subject a VI 1-s schedule of saline injections was used.
Table 2
Sequence of histamine-punishment conditions with variations in VI schedule parameter
Lever 1 Lever 2 Sessions
Schedule 1 Schedule 2 dose (mg/kg) Schedule 1 PM01Histamine PM02 PM03 PM04 PM05 PM06 PM16
VI 120 (pellet) VI 7.5 (inj) 0.3 Extinction 90-93 91-93 89-91 90-93 88-90 VI 120 (pellet) VI 7.5 (inj) 0.3 Avoidance 94-98 94-96 92-94 94-98 91-93 VI 120 (pellet) VI 4 (inj) 0.3 Extinction 97-99 95-97
VI 120 (pellet) VI 4 (inj) 0.3 Avoidance 90-93 100-105 98-103 VI 120 (pellet) VI 2 (inj) 0.3 Avoidance 94-97 99-100 106-110 104-107 99-103
VI 120 (pellet) VI 1 (inj) 0.3 Avoidance 98-109 103-107* 111-124* 108-119 104-124* 27-33
Note.Details are as described for Table 1.
maintained after the exchange in contingen-cies, particularly in one subject (PM01), saline injections were substituted for histamine injec-tions to assess whether those response rates were maintained by the cancellation of hista-mine injections.
Results
During the VI 120-s schedule of food pre-sentation (with the VI 15-s schedule of saline injection) response rates for most subjects sta-bilized at about 10 to 15 responses per min. Response rates for PM02 stabilized at about 6 to 7 and for PM16 at about 25 responses per min (Fig. 1, squares on thefirst panels labeled
“Sal”) and generally occurred at a stable rate
across the session. Response rates on the even-tual avoidance lever stabilized at about 0.5 to 1.5 responses per min (Fig. 1, triangles).
With the introduction of response-produced histamine injections on the punishment lever, response rates decreased to between 50% (PM02 and PM04) and 82% (PM06) of rates obtained with response-produced saline injec-tions (Fig. 1, histamine panels). During subse-quent sessions, response rates recovered for every subject to different extents, though sup-pression remained between 30% (e.g., PM02) and 63% (e.g., PM01) of the response rates prior to the introduction of histamine injec-tions. With these different amounts of response suppression there were no substan-tial differences in the overall rate of food
pellet delivery. The average of 29 (0.4) pel-lets per session decreased by 1.83 (1.7) when histamine injections stably suppressed responding.
With the introduction of the avoidance con-tingency, the rate of responding on the pun-ishment lever did not change systematically across subjects (Fig. 1). Response rates on the avoidance lever remained at low levels for all subjects throughout this phase with the excep-tion of some transient responding during the first two sessions for PM04 and PM06. The mean frequency of histamine injections did not substantially change from that obtained in the previous phase without the avoidance contingency.
Replacing histamine with saline increased responding on the punishment lever for all subjects except PM04 (Fig. 1, second saline panels). With PM02 the initial increase was not sustained in subsequent sessions. The rein-troduction of histamine injections decreased response rates on the punishment lever (Fig. 1, second histamine panels). The decrease was only transient with PM02, which was followed by a small increase when saline was introduced once again (Fig. 1; third saline panel). Throughout all of these subsequent conditions rates of responding on the avoid-ance lever remained low and not appreciably different across conditions (Fig. 1).
In an attempt to increase the rate of respond-ing on the avoidance lever, the punishment and avoidance VI parameter was systematically Table 3
Sequence of procedures for the exchange of contingencies between the levers
Lever 1 Lever 2 Sessions
Schedule 1 Schedule 2
Histamine dose
(mg/kg) Schedule 1 Schedule 2 PM01 PM02 PM03 PM04 PM05 PM16
VI 120 (pellet) VI 1 (inj) 0.3 Avoidance 107-109 105-107 122-124 117-119 122-124 31-33
Avoidance - 0.3/0 VI 120
(pellet) VI 1 (inj) 110-118119-124*
108-123 125-135 120-128 125-127 34-35
VI 120 (pellet) VI 1 (inj) 0.3/0.56 Avoidance - 136-146 129-134 135-138*
128-132 37-41
Avoidance - 0.3/0.56 VI 120
(pellet) VI 1 (inj) 125-131 147-152 139-143
* 133-140 42-49
VI 120 (pellet) VI 1 (inj) 0.3/0.56 Avoidance - 153-168 144-146* 141-149 50-56
Avoidance - 0.3 VI 120
(pellet) VI 1 (inj) 150-157
Note.Details are as described for Table 1.
varied in the next series of conditions (Fig. 2). The left-most disconnected points show response rates on the two levers and numbers of injections when saline injections were sched-uled with and without the avoidance contin-gency under a VI 15-s schedule. Rates of responding on the punishment lever were decreased below saline levels with histamine injections scheduled under a VI 15-s schedule providing a nominal rate of four injections/ min (Fig. 2, top panel). Rates decreased further at a nominal rate of nine injections/min (VI 7.5-s), with further increases in scheduled frequency having no greater effect on response rates. Response rates on the punishment lever were not appreciably different with or without the avoidance contingency (Fig. 2, top, com-pare squares and circles).
Response rates on the avoidance lever were uniformly lower across all VI parameters with histamine injections than with vehicle injec-tions (Fig. 2, middle panel). Rates of respond-ing on the avoidance lever were not different with or without the histamine avoidance con-tingency (Fig. 2, middle panel) and did not vary appreciably with changes in the nominal injection rate. The frequency of histamine injection (Fig. 2, bottom panel) was decreased compared to the frequency of vehicle injec-tions, though that frequency was not different with or without the avoidance contingency (Fig. 2, bottom panel).
Because response rates on the avoidance lever were uniformly low at the introduction of the avoidance contingency, there were fewer opportunities for behavior to come in Fig. 1. Rates of responding on the punishment lever (squares), on the avoidance lever (triangles), and rates of injec-tions (circles) under various schedule condiinjec-tions as described in Table 1.Sal–Punishment lever: Conjoint VI 120-s
(pel-lets), VI 15-s (saline injections), Avoidance lever: EXT; Hist –Punishment lever: Conjoint VI 120-s (pellets), VI 15-s
(0.3 mg/kg histamine injections), Avoidance lever: EXT;Avoid–Punishment lever: Conjoint VI 120-s (pellets), VI 15-s
contact with the avoidance contingencies than there would have been had those rates been higher. To assess the potential of a greater ini-tial incidence of actual avoidances of hista-mine punishment on the maintenance of responding on the avoidance lever, the contin-gencies on the two levers were exchanged.
Response rates on the two levers during the first session after their contingencies were exchanged were similar (Fig. 3). Additionally,
the frequencies of histamine injections during thefirst sessions were uniformly reduced com-pared to the frequencies in the sessions before the exchange. The similar rates of responding on both levers in thefirst session were followed by a progressive decrease in response rates on the avoidance lever across the next several ses-sions, which, except for PM01 and PM02, reached asymptotes that approximated those before the exchange of contingencies, though above zero. Further, the frequencies of hista-mine injections uniformly increased across ses-sions following the change to those obtained before the exchange of contingencies. Similar effects were obtained with PM03, PM04, PM05 and PM16 with subsequent exchanges of the contingencies scheduled on the two levers. In all of the 19 instances across subjects in which the contingencies on the two levers were exchanged, response rates on the avoidance lever were increased, and in all but three instances the number of injections delivered in thefirst session was below that obtained in the immediately preceding session.
For PM01 the response rate on the avoid-ance lever stabilized at a rate above that obtained before the reversal of contingencies on the two levers. To assess if avoidance of his-tamine was maintaining the low response rates on the avoidance lever, histamine injections were replaced with saline for this subject
Fig. 2. Legend on next coloumn.
Fig. 2. Rates of responding on the punishment lever (top panel), on the avoidance lever (central panel), and rates of injections (bottom panel) with variations in the scheduled rate of histamine injection (punishment VI schedule parameter) with (squares) or without (circles) the VI 15-s histamine cancellation contingency in effect. The various schedule conditions are as described in Table 2. Disconnected points above“Veh”represent rates
when saline was injected rather than histamine (0.3 mg/ kg/injection). Data points correspond to the average of all subjects exposed to the condition, with the values for indi-vidual subjects obtained from averages of the last three ses-sions from each condition and error bars corresponding to the standard error of the mean. Statistical analysis of the rate of punished responses indicated significant effect of scheduled injection rate (F= 21.0; p< .001), but no effect of the presence of the avoidance contingency (F= 0.018; p= .894). Statistical analysis of the rate of avoidance responses indicated significant effect of sched-uled injection rate (F= 9.96; p< .001), but no effect of the presence of the avoidance contingency (F= 2.82;
(Fig. 3, PM01 “Saline” panel). Response rates
on both the punishment and avoidance levers increased with saline substitution for hista-mine. Response rates decreased again when histamine was reintroduced.
For PM04 after the second exchange of con-tingencies the histamine dose was increased to 0.56 mg/kg/injection. Although the increase in dose reduced responding on the punish-ment lever, the low rates on the avoidance lever were not appreciably affected. Further exchanges of contingencies, with the increased dose, did not result in acquisition of respond-ing on the avoidance lever (Fig. 3, PM04, three rightmost panels). A single session decrease in dose for PM04 (Fig. 3, session denoted with an asterisk) increased rates of responding on the punishment lever and the frequency of injec-tions. During that session response rates on the avoidance lever decreased.
Cumulative records of performances shown in Figure 4 are from one subject (PM04) that showed a progressive decrease in response rates on the avoidance lever after the
exchange and one subject (PM01) that showed a continued maintenance of respond-ing over successive sessions after the exchange in contingencies on the two levers. The top panel for each column shows records from the last session before the exchange of contingen-cies, with high rates of responding on the pun-ishment lever (the record with tick marks) and low rates on the avoidance lever (the record without tick marks). The middle panel shows thefirst session with the exchanged con-tingencies and the increase in rates of responding on what was in that session for the first time the avoidance lever. The increase in responding on the avoidance lever was most pronounced initially and decreased as the ses-sion progressed. The lower density of tick marks on the event line for the records dis-played in the middle panel shows the reduc-tion in the rate of histamine injecreduc-tion produced by the increased responding on the avoidance lever. The bottom panels show per-formances in the ninth (PM01) and the third sessions after an exchange of contingencies Fig. 3. Effects of exchanges of contingencies between the levers on response rates and injection frequencies over con-secutive sessions. Schedules: Punishment lever: Conjoint VI 120-s (pellets), VI 1-s (0.3 mg/kg histamine injections), Avoidance lever: VI 15-s (histamine injection cancellation).BL:baseline, corresponds to the last three sessions before thefirst exchange of contingencies. Saline panel for PM01 indicates sessions in which histamine was replaced by saline. Under all other conditions, except as noted, histamine injections were 0.3 mg/kg/injection. For PM04 the histamine dose was 0.56 mg/kg/injection for sessions shown in the panel labelled“0.56.”That dose was used during subsequent
(PM04) when response rates for PM04 on the avoidance lever approached those observed before the exchange in contingencies, whereas those for PM01 remained substantially above those before the exchange in contingencies, particularly at the beginning of the session.
Discussion
the injections. Further, when saline replaced histamine injections, rates of responding recovered to those similar to, or approaching, those obtained prior to the introduction of histamine injections. For instances in which the recovery of responding was incomplete, the reintroduction of histamine produced an immediate suppression that was either tran-sient or sustained over subsequent sessions. Further, the number of food pellets received under the VI 120-s schedule did not substan-tially change, suggesting that the suppression of responding was due to punishment by hista-mine injections and not due to a decrease in frequency of reinforcement.
One possible problem when using drugs as punishing stimuli is the potential for “direct”
effects of the accumulated drug to interfere with continued responding. Direct effects refer to those effects of drugs on behavior that occur independently of the contingency rela-tion between the response and the drug
injec-tion. These effects may complicate
interpretations of the effects of the contin-gency between responses and injections, and traditionally have been of concern in studies of the reinforcing effects of drugs (for a detailed discussion see Katz, 1989), but may also apply in the present studies.
Absent a comparison of response-dependent and -independent scheduling of histamine injections, several considerations argue that the effects of histamine in this and previous studies were due to the contingent relation between responses and injections rather than a direct effect of histamine. In the initial stud-ies (Goldberg, 1980; Katz & Goldberg, 1986) of punishment with histamine injections using monkeys, intercalated periods in which responding was not punished were scheduled, with little to no evident effect of accumulated injections on nonpunished responding. Fur-ther, in situations in which histamine sup-pressed food-reinforced responding in rats, concurrent nonpunished responses main-tained by food reinforcement were emitted reliably showing no evidence of suppression (Podlesnik & Jimenez-Gomez, 2013; Podlesnik et al., 2010). A study of histamine injected intraventricularly found a decrease in rates of conditioned avoidance responding (Tasaka, Kamei, Akahori, & Kitazumi, 1985). However, a study of the histamine precursor, L-histadine injected systemically, found no effect on
avoidance responding (Gerald & Maickel, 1972). Another study found that, unlike punishing effects, the effects of centrally administered histamine were insensitive to administration of histamine antagonists of either subtype (Calcutt & Reynolds, 1976), indicating that the effect was a by-product of central administration and, in contrast to pun-ishing effects (e.g. Goldberg, 1980; Katz & Goldberg, 1986; Podlesnik & Jimenez-Gomez, 2013), was not mediated by histamine recep-tors. Taken together, the results to date sug-gest virtually no direct effects of systemically administered histamine.
It is interesting to note that in the present study histamine was effective as a punishing stimulus in suppressing behavior but ineffec-tive in establishing an avoidance response. The absence of negative reinforcing effects in this specific situation highlights that conse-quences that suppress behavior do not by necessity function to maintain behavior that avoids their presentation. This finding is consistent with an emphasis on the situation-specific nature of the effects of behavioral con-sequences. As noted by Morse and Kelleher (1977), there is a tendency to emphasize the consequent event and to ignore the impor-tance of the contingency relationships that determine subsequent behavior. Effects of many routinely studied environmental events are often presumed, though it is preferable to base categorizations of stimulus events on the manner in which they change behavior.
punishing stimuli, such as electric shock, used in situations in which responses are reliably elicited.
In another variant of a two-factor hypothe-sis, responsesother than the punished responseare negatively reinforced if this emission competes with the punished response, thereby decreas-ing the frequency of the punishdecreas-ing stimulus, or stimuli paired with it (e.g. Dinsmoor, 1954; Skinner, 1953; Solomon, 1964). Despite pun-ishing effects observed on introduction of his-tamine injections in the present study, avoidance responding on a second lever was not stably maintained at levels greater than those obtained with vehicle. With the initial introduction of the avoidance contingency the rate of responding on the avoidance lever was exceedingly low, with no acquisition of that response evidenced over subsequent sessions.
With the contingencies on the levers
exchanged, response rates on the avoidance lever were initially sufficient for contact with the avoidance contingencies, and the fre-quency of histamine injection initially decreased. However, over the course of succes-sive sessions, responding on the avoidance lever extinguished. Further, repetitions of the exchange were no more effective in producing a sustained increase in avoidance responding, and an increase in histamine dose for PM04 did not increase rate of responding on the avoidance lever. Thus, the conditions which were sufficient to produce suppression were not sufficient for the maintenance of avoid-ance responding on a second lever, suggesting that the observed punishment suppression was not dependent upon the acquisition of the avoidance response.
It remains possible that some unobserved avoidance response was conditioned before the introduction of the explicitly programmed avoidance contingency, conserving a two-factor interpretation of the punishing effects of histamine. Casual observations did not indi-cate any such responses, with that identifi ca-tion being an essentialfirst step for testing this version of the hypothesis. When the other response is unspecified, or is as ill-defined as
“any behavior other than the punished
response,” the hypothesis both lacks
parsi-mony and is untestable. If a prior avoidance response had been established it would unlikely be as efficient in reducing the fre-quency of histamine injections as responses on
the avoidance lever, especially when contin-gencies on the two levers were exchanged. As shown, those elevated response rates on the avoidance lever, rather than being maintained, extinguished over successive sessions.
The present procedure by design omitted a change in exteroceptive stimulus conditions contingent on effective avoidance responses. A similar tack was taken by Arbuckle and Lattal (1987) in which punishment cancellation was contingent on a period of response omission, which resourcefully exploits its compatibility with operant response suppression. Conse-quently, no specific training was necessary for maintenance of the avoidance response, and the results were consistent with a two-factor hypothesis of punishment. The present study similarly programmed a punishment avoid-ance contingency for a specific response, though on a separate lever (cf. Catania, 1973). It is possible that some as yet unidentified aspect of the experimental situation specifi -cally interfered with acquisition of lever-press avoidance of punishment. Mitigating against this interpretation is that several previous stud-ies have demonstrated the maintenance of one response with avoidance of consequences arranged for a separate response (e.g., Morse & Herrnstein, 1956; van Haaren & Zarcone, 1994; Thomas, 1968). Further, escape from histamine injections previously has been successfully trained and maintained in rhesus monkeys under different conditions (Takada et al., 1986). Possibly most important is that when the contingencies on the levers were exchanged in the present study, respond-ing occurred on the avoidance lever at rates sufficient to decrease the frequency of hista-mine injections, indicating that the two responses were not incompatible.
rates. Had responding been maintained by avoidance of histamine injections, adecrease in response rates would be expected when saline was substituted for histamine. These results suggest the possibility that the responding of PM01 on the avoidance lever was adventi-tiously maintained, possibly through chaining of the responses on the two levers.
Further evidence contrary to the two-factor hypothesis would have been provided if avoid-ance of histamine injections had been trained alone prior to the implementation of punish-ment, and not subsequently maintained solely by avoidance of histamine punishment. On one hand, that procedure could demonstrate only that histamine punishment and avoid-ance can occur concurrently, but would not speak to the necessary and sufficient condi-tions for punishment. On the other hand, punishment in the absence of avoidance is one true logical test of the avoidance account of punishment.
In summary, the present study found effec-tive punishment by histamine injection with-out evidence of avoidance of the punishing stimulus. As in previous studies (Azrin et al., 1965; Lattal & Cooper, 1969) an added stimu-lus change may have facilitated acquisition of the avoidance response under the current conditions. The present study, however, addresses whether the decrease in frequency of the punishing stimulus alone is sufficient for the maintenance of avoidance responding. The present results indicate that conditions sufficient for histamine punishment were not sufficient for the maintenance of an explicit punishment-avoidance response, an outcome indicating the absence of a necessary general relation between avoidance and histamine punishment.
References
Arbuckle, J. L., & Lattal, K. A. (1987). A role for nega-tive reinforcement of response omission in punish-ment. Journal of the Experimental Analysis of Behavior,
48(3), 407–416. https://doi.org/10.1901/jeab.1987.
48-407
Azrin, N. H., Hake, D. F., Holz, W. C., & Hutchinson, R. R. (1965). Motivational aspects of escape from punish-ment. Journal of the Experimental Analysis of Behavior,
8(1), 31–44. https://doi.org/10.1901/jeab.1965.8-31
Azrin, N. H., & Holz, W. C. (1966). Punishment. In W. K. Honig (Ed.), Operant behavior: Areas of research and application (pp. 380–447). New York:
Appelton-Century-Crofts.
Barker, D. J., Sanabria, F., Lasswell, A., Thraikill, E. A., Pawlak, A. P., & Killeen, P. R. (2010). Brief light as a practical aversive stimulus for the albino rat.Behavior and Brain Research, 214(2), 402–408. https://doi.
org/10.1016/j.bbr.2010.06.020
Branch, M. N., Nicholson, G., & Dworkin, S. I. (1977). Punishment-specific effects of pentobarbital: Depen-dency on the type of punisher.Journal of the Experimen-tal Analysis of Behavior, 28(3), 285–293. https://doi.
org/10.1901/jeab.1977.28-285
Calcutt, C. R., & Reynolds, J. (1976). Some behavioural effects following intracerebroventricular (ICV) injec-tion in rats of histamine H 1- and H 2-receptor ago-nists and antagoago-nists.Neuroscience Letters,3(1), 82–83.
https://doi.org/10.1016/0304-3940(76)90112-9 Carvalho Neto, M. B., Maestri, T. C., Tobias, G. K. da S.,
Ribeiro, T. C., Coutinho, E. C. N. N., Miccione, M. M.,…Moreira, D. (2005). O jato de ar
quente como estímulo punidor em rattus norvegicus.
Psicologia: Teoria e Pesquisa, 21(3), 335–339. https://
doi.org/10.1590/S0102-37722005000300010
Catania, A. C. (1973). Self-inhibiting effects of reinforce-ment. Journal of the Experimental Analysis of Behavior,
19(3), 517–526. https://doi.org/10.1901/jeab.1973.
19-517
Catania, A. C. (2013). Learning (5th ed.). Cornwall-on-Hudson, NY: Sloan Publishing.
Church, R. M. (1969). Response suppression. In B. A. Campbell & R. M. Church (Eds.), Punishment and aversive behavior(pp. 111–156). New York:
Apple-ton-Century-Crofts.
de Villiers, P. A. (1974). The law of effect and avoidance: A quantitative relationship between response rate and shock-frequency reduction.Journal of the Experimental Analysis of Behavior, 21(2), 223–235. https://doi.
org/10.1901/jeab.1974.21-223
Dinsmoor, J. A. (1954). Punishment: I The avoidance hypothesis.Psychological Review,61, 34–46. https://doi.
org/10.1037/h0062725
Dinsmoor, J. A. (1998). Punishment. In W. T. O’Donohue
(Ed.), Learning and behavior therapy (pp. 188–204).
Needham Heights, MA: Allyn & Bacon.
Estes, W. K. (1944). An experimental study of punishment.
Psychological Monographs,57, 1–40. https://doi.org/10.
1037/h0093550
Everly, J. B., & Perone, M. (2012). Suppressive and facilita-tive effects of shock intensity and interresponse times followed by shock.Journal of the Experimental Analysis of Behavior, 98(3), 311–340. https://doi.org/10.1901/
jeab.2012.98-311
Ferster, C. B. (1958). Control of behavior in chimpanzees and pigeons by time out from positive reinforcement.
Psychological Monographs: General and Applied, 72(8), 1–38. https://doi.org/10.1037/h0093787
Flaherty, C. F. (1985). Animal learning and cognition. New York: McGraw-Hill, Inc.
Fleshler, M., & Hoffman, H. S. (1962). A progression for generating variable-interval schedules. Journal of the Experimental Analysis of Behavior, 5(4), 529–530.
https://doi.org/10.1901/jeab.1962.5-529
Friedel, J. E., DeHart, W. B., & Odum, A. L. (2017). The effects of 100 dB 1-kHz and 22-kHz tones as punishers on lever pressing in rats. Journal of the Experimental Analysis of Behavior, 107(3), 354–368. https://doi.
Galbicka, G., & Branch, M. N. (1981). Selective punish-ment of interresponse times.Journal of the Experimental Analysis of Behavior, 35(3), 311–322. https://doi.
org/10.1901/jeab.1981.35-311
Gerald, M. C., & Maickel, R. P. (1972). Studies on the pos-sible role of brain histamine in behaviour.British Jour-nal of Pharmacology, 44, 462–471. https://doi.org/10.
1111/j.1476-5381.1972.tb07284.x
Goldberg, S. R. (1980). Histamine as a punisher in squirrel monkeys: Effects of pentobarbital, chlordiazepoxide and H1- and H2-receptor antagonists on behavior and cardiovascular responses.The Journal of Pharmacol-ogy and Experimental Therapeutics,214(3), 726–736.
Herrnstein, R. J. (1955). Behavioral consequences of the removal of a discriminative stimulus associated with variable-interval reinforcement. Unpublished doctoral dis-sertation, Harvard University.
Herrnstein, R. J., & Hineline, P. N. (1966). Negative rein-forcement as shock-frequency reduction.Journal of the Experimental Analysis of Behavior, 9(4), 421–430.
https://doi.org/10.1901/jeab.1966.9-421
Holtz, N. A., & Carroll, M. E. (2015). Cocaine self-administration punished by intravenous histamine in adolescent and adult rats. Behavioural Pharmacology,
26(4), 393–397. https://doi.org/10.1097/FBP.000000
0000000136
Holz, W. C., & Azrin, N. H. (1962). Recovery during pun-ishment by intense noise. Psychological Reports, 11, 655–657. https://doi.org/10.2466/pr0.1962.11.3.655
Katz, J. L. (1989). Drugs as reinforcers: Pharmacological and behavioral factors. In J. M. Liebman & S. J. Cooper (Eds.), The Neuropharmacological Basis of Reward(pp. 164–213). Oxford, UK: University Press.
Katz, J. L., & Goldberg, S. R. (1986). Effects of H1-receptor antagonists on responding punished by his-tamine injection or electric shock presentation in squirrel monkeys.Psychopharmacology,90(4), 461–467.
https://doi.org/10.1007/bf00174061
Koffarnus, M. N., & Winger, G. (2015). Individual differ-ences in the reinforcing and punishing effects of nico-tine in rhesus monkeys. Psychopharmacology, 232, 2393–2403.
https://doi.org/10.1007/s00213-015-3871-8
Kuraishi, Y., Nagasawa, T., Hayashi, K., & Satoh, M. (1995). Scratching behavior induced by pruritogenic but not algesiogenic agents in mice.European Journal of Pharmacology, 275, 229–233. https://doi.org/10.
1016/0014-2999(94)00780-B
Lattal, K. A., & Cooper, A. M. (1969). Escape from punish-ment by omission of responding.Psychonomic Science,
15(5), 263–264. https://doi.org/10.3758/BF03336341
Leitenberg, H. (1965). Is time-out from positive reinforce-ment an aversive event? A review of the experireinforce-mental evidence. Psychological Bulletin, 64(6), 428–441.
https://doi.org/10.1037/h0022657
Masserman, B. J. H., & Pechtel, C. (1953). Neuroses in monkeys: A preliminary report of experimental obser-vations.Annals of the New York Academy of Sciences,56, 253–265. https://doi.org/10.1111/j.1749-6632.1953.
tb30221.x
Morse, W. H., & Herrnstein, R. J. (1956). The mainte-nance of avoidance behavior using the removal of a conditioned positive reinforcer as the aversive stimu-lus. American Psychologist, 11(8), 430. https://doi. org/10.1037/h0041236
Morse, W. H., & Kelleher, R. T. (1977). Determinants of reinforcement and punishment. In W. K. Honig & J. E. R. Staddon (Eds.), Handbook of operant behavior. (pp. 174–200). Englewood Cliffs, NJ: Prentice Hall.
Negus, S. S. (2005). Effects of punishment on choice between cocaine and food in rhesus monkeys. Psycho-pharmacology, 181(2), 244–252. https://doi.org/10.
1007/s00213-005-2266-7
Perone, M., & Galizio, M. (1987). Variable-interval sched-ules of time out from avoidance.Journal of the Experi-mental Analysis of Behavior,47(1), 97–113. https://doi.
org/10.1901/jeab.1987.47-97
Podlesnik, C. A., & Jimenez-Gomez, C. (2013). Punishing and cardiovascular effects of intravenous histamine in rats: Pharmacological selectivity.Journal of the Experi-mental Analysis of Behavior, 100(3), 333–354. https://
doi.org/10.1002/jeab.46
Podlesnik, C. A., Jimenez-Gomez, C., & Woods, J. H. (2010). A choice procedure to assess the aversive effects of drugs in rodents.Journal of the Experimental Analysis of Behavior, 93(2), 203–223. https://doi.
org/10.1901/jeab.2010.93-203
Prada, J. A., Takada, K., Katz, J. L., Goldberg, S. R., & Barrett, J. E. (1987). Punishment of behavior with buspirone and gepirone in the squirrel monkey. Feder-ation Proceedings, 46(4), 1300.
Sharpless, S. K. (1961). Effects of intravenous injections of epinephrine and norepinephrine in a choice situa-tion.Journal of Comparative and Physiological Psychology,
54, 103–108. https://doi.org/10.1037/h0049243
Sidman, M. (1953). Two temporal parameters of the main-tenance of avoidance behavior by the white rat. Jour-nal of Comparative and Physiological Psychology, 46(4), 253–261. https://doi.org/10.1037/h0060730
Skinner, B. F. (1938).The behavior of organisms: An experi-mental analysis. New York, Appleton-Century-Crofts, Inc.
Skinner, B. F. (1953). Science and human behavior. New York/London: Free Press/Collier MacMillan. Solomon, R. L. (1964). Punishment.American Psychologist,
199, 239–253. https://doi.org/10.1037/h0042493
Spealman, R. D. (1978). Pressurized air as a punisher. Jour-nal of the Experimental AJour-nalysis of Behavior, 2(29), 341–345. https://doi.org/10.1901/jeab.1978.29-341
Takada, K., Barrett, J. E., Allen, M. S., Cook, J. M., & Katz, J. L. (1992). Punishment of schedule-controlled behavior with beta-carboline injections: Antagonism and comparisons with other compounds.The Journal of Pharmacology and Experimental Therapeutics, 261(1), 138–145.
Takada, K., Winger, G., Cook, J., Larscheid, P., & Woods, J. H. (1986). Discriminative and aversive prop-erties of β-carboline-3-carboxylic acid ethyl ester, a
benzodiazepine receptor inverse agonist, in rhesus monkeys. Life Sciences, 38, 1049–1056. https://doi.
org/10.1016/0024-3205(86)90240-7
Tasaka, K., Kamei, C., Akahori, H., & Kitazumi, K. (1985). The effects of histamine and some related com-pounds on conditioned avoidance response in rats.
Life Sciences, 37(21), 2005–2014. https://doi.org/10.
1016/0024-3205(85)90031-1
Thomas, J. R. (1968). Fixed-ratio punishment by timeout of concurrent variable-interval behavior.Journal of the Experimental Analysis of Behavior, 11(5), 609–616.
Van Haaren, F., & Zarcone, T. (1994). Some functional characteristics of avoidance of timeout from response-dependent food presentation in rats.Behavioural Pro-cesses, 31, 197–206.
https://doi.org/10.1016/0376-6357(94)90006-X
White, J. M., & Rumbold, G. R. (1988). Behavioural effects of histamine and its antagonists: A review. Psychophar-macology, 95(1), 1–14. https://doi.org/10.1007/BF0
0212757
Woolverton, W. L., Freeman, K. B., Myerson, J., & Green, L. (2012). Suppression of cocaine self-administration in monkeys: Effects of delayed punish-ment. Psychopharmacology, 220(3), 509–517. https://
doi.org/10.1007/s00213-011-2501-3.
