Image Transfer

pixel

initiation y select y request

initiation x

select x

data acknowledge

2 8

\

~I---'r.

---~~ "~----4---

3

Figure 3.15: Timing diagram for data transfer between sender and receiver.

select y

request

5V select x

data valid

acknowledge

reset y

5 6 7 8 9 10 11 12 13 14 Time (microseconds)

Figure 3.16: Data transfer between sender and receiver. Data taken from real system. [Not

all the signals shown in the t.iming diagram (Figure 3.15) are instrumented on the chips]. Data were collected with a digital scope triggering off of the falling edge of the x-select signal and are synchronized t.o that. The chip was configured in such a way that all the neurons were firing at a high rate, in order to measure the minimum data transfer period. The minimum period was approximately 2· 10-⁶ seconds.

column ack

P-~

^tall

r

xdecode

1 1

^delta

^-7-1 ~!

I

^scanout

~ y-decode --->-1-++---1

Figure 3.17: Schematic of single node in the receiver array.

o

5 10 15 20 25 30 35 40 45

Time (microseconds)

Figure 3.18: The receiving element generates a step in potential in response the arriving address-event. The voltage step size was arbitrarily scaled by the off-chip current-sensing amplifier. The bottom trace is the Acknowledge signal that terminates the data transfer. The Acknowledge signal is 5 volts in amplitude and 1 microsecond in duration.

The behavior of the receiving pixel is illustrated in Figure 3.19. Synthetic data in the form of a temporal stream of digital addresses were generated on an HP 9836C workstation and transmitted to the chip with a custom hardware interface board. The time constant of integration on the receiver determines the time over which spikes can be averaged. This time constant of integration, controlled by tau, can be varied over several orders of magnitude.

A long integration time is advantageous for integrating a small signal that is contaminated with sporadic randOlTI noise. A short integration time increases the temporal resolution of the system and a simple threshold is able to detect spike coincidence to within the integration time of the integrator.

This receiving pixel should be modified to include a leak whose magnitude is a function of the voltage level of the integrator. This feature would allow a stable translation of event frequency into analog voltage level. Additional circuitry may be necessary to control better the quantity of input charge for each event. In the existing design, the event duration is linearly related to the amount of charge that is deposited on the integration capacitor for a given event. Events of different duration will result in different amounts of current being integrated on the capacitor. One solution to this problem would be to put a timing element in each pixel that regenerated a long-duration spike, triggered by the event. If such a long- duration spike tnechanislll were incorporated the fractional va.riation in event width would be caused by transistor rniSlnatch on the receiving chip rather than transmission variability.

A longer spike would presumably have less fractional duration variation. However, this solution requires lllorc area. Event durations do not appear to vary by more than 50 percent.

Until it can be demonstrated that there is a significant impact on the computation, there is no reason to include such a mechanism.

The first step in image transfer is the creation of the image on the retina. The retinal pixel incorporated into the self~timed data transfer system generates events when the light level increases. In this way, it is similar to the on-transient retinal ganglion cell [7]. A schematic diagram of the pixel circuitry is shown in Figure 3.20. The drive to the spike- generating pixel generated by a circuit similar to that of the feedforward retina described in the previous chapter, but the resistors have been omitted to reduce the size of the pixel.

The drive circuit averages the output of the logarithmic photoreceptor with a follower- integrator whose tilne consta.nt is controlled by tau. The output of the drive circuit is a

Long integration time

Short integration time

Data valid

o

5 10 15 20 25 30 35 40 45

Time (milliseconds)

Figure 3.19: The output of a single pixel on the receiver chip shown for two different inte- gration times. Address-events were generated with a custom hardware interface board and an HP9836C computer for the receiver pixel accessed by the serial scanner. The integration time of the pixel was modified by changing the bias voltages on the tau and delta controls.

Level was set to 4.046 volts. For the fast integration time trace, delta was set to 0.67 volts and tau was set to 2.756 volts. For the slow integration time trace, delta was set to 0.62 Yolts, and tau was set to 2.85 volts. See Figure 3.17.

select x initiate x

~ refractory '<---leak

Figure 3.20: Schematic of a single pixel in the sender array.

select y

initiate y

current proportional difference between the average intensity and the instantaneous values of intensity, which is scaled by the control voltage i. This system is analogous to the bipolar cell of the outerplexiform layer of the retina. The spike-generation circuitry of the pixel is like that shown in Figure 3.10, except that data transfer sequence takes place in two dilnensions. The primary state variable, eN, is like the Inembrane capacitance of a retinal ganglion cell. This capacitor is a leaky integrator with a time-constant set by the leak parameter, which determines the quiescent voltage on CR. Capacitor CN integrates the charge supplied by the drive circuitry until its voltage reaches the inverter threshold. The inverter initiates the data transfer process. Once the pixel is selected in both the x- and y-dimensions, capacitor

eN

is discharged.

The parameter settings of the pixel affect the number of spikes that it generates in response to a particular stimulus. The responses of a pixel to a flashing LED for several different settings of the time-constant of the differentiator are shown in Figure 3.21. If the follower integrator is able to follow the stimulus intensity more quickly, less current is produced by the differencing amplifier and so fewer spikes are produced.

Figure 3.22 shows the clift·erence in response caused by the refractory period, which is analogous to the duration of the delayed-rectifier current, 1Ko' in biological neurons. Like

tau=0.423V

tau=0.454V

tau=0.475V

o

5 10 15 20 25 30 35 40 45 Time (milliseconds)

Figure 3.21: The response of the complete 8ender~receiver system to a flashing light-emitting diode (LED) of intensity 63.2 UlW /mml with three different time-constants for the differen- tiator. Stimulus onset is indicated by a vertical line. The output of the sending pixel and the corresponding node on the receiver arc shown 8S a pair, the sender pixel waveform above the receiver response. Responses were averaged by the digital oscilloscope over eight stimulus presentations. The voltage, lint, controlling the time constant of the differcntiator in the send- ing pixeJ is shown next to each pair of responses. All other parameters were held constant.

o

⁵

refractory=3.20

refractory=3.27

rcfractory=3.34

10 15 20 25 30 35 40 45 Time (milliseconds)

Figure 3.22: The output of a single pixel on the sender chip and the corresponding node on the receiving chip. Stimulus was a flashing LED of intensity 63.2 mWJmm²• Stimulus onset is indicated by a vertical line. Values of the refractory transistor gate voltage are shown next to each pair of responses. As the refractory period decreases, the maximum event rate increascs so the number of events per stimulus presentation increases.

the 1"0 current, the reset current is sensitive to the voltage of the pixel. The reset current increases in amplitude until the voltage on capacitor CN is discharged below the inverter threshold. When the Acknowledge has reset either the x- or y-initiation node, the select signals that are contributing to the reset current arc withdrawn. The magnitude of the reset current set by the voltage on capacitor CR decays at a rate set by the refractory control.

When the reset current is smaller than the current from the differentiator, the pixel voltage begins to increase. When the reset current is of longer duration, the voltage on the pixel capacitor rClnains low longer. Fewer spikes arc produced in response to the same stimulus and thus there is less activity in the receiving node.

The gain of the action-potential is generated in biological neurons by the positive feed-

back from the sodium spike channels. The sodium phase of the action potential is generated by the digital circuitry on the chip. Even with the gain of the inverter, the parameter i that scales the difference current driving the state capacitor must not be too small, or else the inverter will not cross threshold quickly enough for the data transfer process to proceed quickly. Several of the inverters in a row may be approaching their transitions and their ef- fects sum to initiate a horizontal request. Once the row is selected, there is a delay until one of the inverters crosses threshold far enough to initiate data transfer in the column. This delay is apparent only when the current flowing into the state capacitor is very small. The gain problem may be ameliorated by incorporating positive feedback from the row select to the pixel. However, all of the pixels along the row would receive this positive feedback. Any such feedback mechanism should be capacitive, so that the feedback cannot be integrated by the initiation mechanism into an entirely new event. The magnitude of the feedback should be small enough not to bring all the pixels in the row past the inverter threshold.

The major drawback to this particular pixel is that it is not sufficiently sensitive with low-offset to make a pra.ctical imager using this cOl1uuunications protocol. The gain of this photoreceptor is low and the DC offsets arc integrated by the pulse generation mechanism so that much of the bandwidth is occupied transmitting offset data. The data that were taken in this chapter were taken with the chip configured to have a large quiescent leak, which reduced DC off·set problems. However, the stimulus needed to be high contrast enough to elicit an above-threshold response.

Figure 3.23 shows the response of the system to increasing intensity steps. The mag- nitude of the step in light intensity is encoded by the number of spikes generated. Event timing as well as total number of events carry information about the image since the latency of response is increased when the stiluuius has lower contrast. A silnilar phenomenon is observed in biological visual systems. It forms the basis of the Pulfrich effect, a stereoscopic depth illusion. Placing a. neutral density filter in front of one eye causes a delayed response to the stimulus from that eye. This delay is interpreted by the motion-interpolation pro- cessing in the cortex as a shift in the position of the target between the two eyes. This artificially induced disparity is indistinguishable from real depth. A pendulum bob swinging back and forth in a plane in front of the viewer is seen to lnove in a circle in depth.

The representation of temporal change is natural for the address-event representation;

126.8mW /mm'

63.2mW /mm'

31.8mW /mm'

12.4m\¥ mm²

o

5 10 15 20 25 30 35 40 45 Time (milliseconds)

Figure 3.23: Analog data from a single sender pixel and the corresponding receiver node to flashing LED of different intensities. Light onset is indicated by a vertical line. The intensity of the flash is shown next to each pair of traces. The number of spikes and the response latency are a function of the step size. The bottom pair of traces shows the response of the sending pixel to a small intensity flash. Current is integrated on the state capacitor, but the pixel fails to reach threshold. The current decays away at a rate set by the leak voltage. In this case, the leak voltage was 0.65 volt.

tClnporai accuracy is inlportant and events are sparse in the retinal array. In general, a delta-modulated encoding of data is best for this communication protocol. The full signal must be represented by changes in the signal, and the effects of these changes integrated by the receiver, if the full DC value is to be reconstructed. For such a reconstruction, the time constant of integration on the receiver should be long. In contrast, the time constant of integration on the receiver should be short for the detection of temporal coincidence of events. Both of these regimes of operation are easily achieved within the range of CUlTent levels in subthreshold CMOS transistors. Both can be done in parallel on the same re- ceiving chip, 01' on different receivers. Different time constants of integration or frequency characteristics are observed in parallel streams of the visual system. The magnocellular system is responsible for transmitting high temporal frequency information and has a lower integration time, while the parvoccllular system is responsible for higher spatial frequencies but with longer integratioll times.

Of course, it is desirable to instrument the entire ituaging array. The retinotopic nature of image transfer is best illustrated by comparing images scanned from the sender and the receiver chip using traditional analog video scanning techniques. The image of a flashing LED as it appears on the sending retina is depicted in Figure 3.24 and the corresponding image on the receiver is shown in Figure 3.25.