Each pixel takes the difference of these intensities (the gradient) and multiplies them to generate precursors of the terms in the characteristic equation C. Left: Data as a function of LCD monitor value, in units used by the monitor.
Vision for Motion Estimation
The local motion estimate can then be combined to calculate the 3D motion of the imaging camera relative to its surroundings. Although the system presented here does not calculate the trajectories of other moving objects, it provides the basis for doing so in the future.
Custom Single-Chip Approach
Medium- and long-range sensors that can handle such tasks include both vision sensors that detect light reflected from objects in the environment, and active sensors that emit and detect their own signal, such as laser rangefinders. Active sensors are not suitable for environments where stealth is important, as the signal emissions can be detected by other agents.
Analog versus Digital Processing
In both of these scenarios, cramming the most expensive compute into a specialized, lower-power processor can be a big win.
Motion Estimation Algorithms
- The Basic Gradient Model for Motion Estimation
- Energy Models for Motion Estimation
- The Kanade Motion Detector
- Token-Based Motion Estimation
- Our Implementation
The terms in the denominator are different order derivatives of the intensity at a single point in the image. Like the Reichardt model, the final output of the Adelson-Bergen motion detector is a comparison of motion in the right and left directions.
Motion Estimation in Hardware
Alternatively, the processing is performed by circuits at the edges of the photodetector array, but still on the same chip. Parallelization of feature extraction, as recommended in this thesis, is a valuable improvement to the design [31] because it reduces the computational demands placed on the digital postprocessor.
Mismatch Between Array Elements
D´ıaz provides tables that allow estimation of the number of gates used and thus the silicon area of the FPGA. Logarithmic compression is analogous to human vision and extends the dynamic range of the photoreceptor far beyond that of a CCD camera [51] or a CMOS integrating photoreceptor.
Beyond Preprocessing
Motion parallel to the edge will result in little change near the central pixel. Even from this small local neighborhood, the motion of the angle in the image can be calculated correctly.
Tomasi Kanade Features
Algebraic rearrangement of the terms and rewriting in matrix form leads to a definition of the characteristic matrixC. In practice, the maximum size of the eigenvalues is bounded by the maximum possible pixel value.
Prior aVLSI Implementation
- Overview of Pesavento’s Chip
- Mismatch Problems in Pesavento’s Chip
- Brief Overview of Floating Gates for Mismatch Reduction
- Reducing Circuit Mismatch in Non-linear Circuits
- Requirements for Mismatch Correction in Pesavento’s Circuits
Six floating gates will compensate for the two degrees of freedom in each of the three Gilbert multipliers. First, the layout area required for all ten of the floating gates would greatly increase the pixel size.
Orthogonal Gradient Detector (OGD)
- Reducing the Kanade Detector
- Definition of the Orthogonal Gradient Detector (OGD)
- Comparison of Kanade detector and OGD
- Briefly, on Modeling
- Mismatch-Invariant Layout
- Mismatch in Arrays
- Reducing Mismatch with Floating Gates
Let us review the intuitive meaning of the equations describing the Canada feature detector (Equations 2.3–2.5). To complete the comparison of the Canada detector and the OGD, I considered how well the detected features can be used for local motion estimation. In a sense, therefore, OGD has better coverage, since the features provide information about more areas of the image than the Canada detector.
The backs of the chairs have a spatial structure that can be well distinguished by the Canada detector.
Pixel Overview
Its output (L) is proportional to the logarithm of the light intensity and is provided as a voltage input to each of its neighbors. A pixel performs two different operations, one to calculate the horizontal gradient (LU P −LDOW N) and the other to calculate the vertical gradient (LLEF T−LRIGHT). Four sample-and-hold (S/H) circuits store the gradient values for the current and previous time steps for each of the two difference calculations.
Because each difference circuit (section 3.4) contains two floating gates, including two copies of the difference circuit.
![Figure 3.5: Gate oxide thickness as a function of time. The date is extracted from two books (by Brown [64] and by Sharma), a talk by Gordon Moore (Intel), and the International Technology Roadmap for Semiconductors [1]](https://thumb-ap.123doks.com/thumbv2/123dok/10401200.0/58.918.300.675.128.450/figure-thickness-function-extracted-international-technology-roadmap-semiconductors.webp)
Photoreceptor Circuit
- Light Detection Devices
- The CMOS Integrating Pixel Sensor
- The CMOS Continuous-Time Logarithmic Photoreceptor
- Mismatch and Calibration
- The Fabricated Logarithmic Floating-Gate Photoreceptor
The capacitance of the NFET and PFET instantiations is provided by the inherent capacitance of the photodiode. In the case of the NFET, the gate voltage Vg can be connected to Vdd for simplicity. That is, they subtract the output from the pixel in response to the photocurrent from the output in response to a calibration current.
Both values for the floating-gate photoreceptor are less than the 2.1mV resolution of the ADC (Analog-to-Digital Converter) in the instrument used to collect the data.
Difference Circuit
Difference Circuit Topology and Analytical Description
I will refer to a shift in the slope of the line as a "gain" mismatch, denoted by γ. In physical terms, this corresponds to a change in the absolute magnitude of the output current. Of the remaining transistors, simulations indicate that mismatch in the input PFETs (P1-P4) is the dominant source of mismatch.
The calibration of the difference circuit can be divided into two stages, the first for fgρ and the second for fgγ.
Difference Circuit Current-to-Voltage Conversion
Since the differential circuits are programmed one at a time, during this calibration phase we temporarily adjust the Vbias if necessary to maintain consistent gain. We then allow the differential circuit to output its current Idif to the capacitor for the set integration time tinteg. Since the output of the differential circuit should be symmetrical around Idi = 0, an intermediate value such as 3.25V is a good choice for Vreset.
All measured results from the difference circuit are buffered by the current-to-voltage converter and sample-and-hold, as described here.
Difference Circuit Measured Results
For each stimulus, contrast varied from zero (white and black areas of equal intensity) to 255 (white and black areas as maximally different as the LCD monitor stimulus allowed). At low contrasts, the LCD monitor area is compressed and the difference circuit does not detect the difference. At high contrasts, the differential circuit saturates because I used a constant-length integration time to convert current to voltage.
Unfortunately, I underestimated the amount of misalignment (VM max) and didn't build in enough offset (Vbi).
Sample-and-Hold Circuit
S/H Design
The input voltage Vin is fed into Vout by the amplifier and buffer combination. Thus, the voltages at most of the amplifier's internal nodes should remain roughly where they were during operation, hopefully reducing leakage from Vhold. Only a much smaller portion of the charge goes to the drain and thus to the holding capacitor.
Most of the remaining pedestal must be the result of parasitic gate-to-drain capacitance at P5 and N7.
S/H Measured Data
Note, however, that if the mismatch between the cascode transistors is greater than the inherent difference in performance between the NFET and the PFET, then the mismatch may contribute more to the base size. The blue lines indicate the output of the sample-and-hold circuit in sample mode. In this region, most of the injected charge must come from the gate-drain capacitive coupling as Vnca and Vpcasswing to Vgnd and Vdd.
Similarly, the value of Vpcas affects the size of the pedestal as shown in fig.
Saliency Circuit
If the gradient values (VgradX and VgradY) are too close to one of the thresholds, the comparator can output a voltage between the rails. The salient output is smoothed by an inverter both to increase the driveability of the circuit and to decrease the range over which the final output of the chip can be in a non-binary value. This method requires the flash circuit to be fast enough to not limit the read speed of the array.
A minimum array readout at 30 Hz and an array of 18 x 18 pixels on this test chip requires a readout time of 100 us per pixel.
Power Supply Sensitivity
Effects of Power Supply Fluctuations on the Photoreceptor
Vfg is the voltage on the floating gate, which allows adjustment of the DC offset of the photoreceptor output. In simulations of the overall impact of power supply fluctuations on the photoreceptor output, I have assumed that VddA only couples to the floating gate via Vphn. The output is much more sensitive to fluctuations on Vphn, which are coupled to the floating gate voltage, Vfg, and change the photoreceptor's operating point.
The range of supply sensitivity was even greater when I swept out the biases associated with the follower.
Effects of Power Supply Variation on Difference Circuit
Vf olis swept by connecting power supply to off-chip resistor source If bias to VddA. Note that the values listed in the table are the onVout change for a 500 mV change in VddA, which is much larger than the actual power supply noise we should reasonably expect. Left: The slope shown here is a measure of the change in the output current Idif f as a function of the differential input V1−V2.
On the currently manufactured chip, they are supplied by independent power supplies and therefore independent of Vdd insofar as laboratory instruments are independent.
Temperature Sensitivity
- Temperature Effects in the Photodiode
- Review of MOS Transistor Temperature Effects
- Impact on Floating-Gate Devices
- Temperature Effects in the Photoreceptor
- Temperature Effects in the Difference Circuit
Changes in the temperature of the photodiode have little effect on the photocurrent and a significant effect on the dark current. The dependence of the MOS capacitance as a function of the gate voltage is shown in Fig. Finally, the capacitance of the floating gate will change with temperature, affecting the stored voltage on the floating gate.
In subthreshold transistors, temperature mainly affects the Io term of the subthreshold current equation (eq. 3.20).
Architecture for Real-Time Processing
To perform calculations on small portions of an image, pixel data can be temporarily stored in on-chip sampling and storage circuits. You would need some additional control circuitry, either on-chip or built into an external controller. Consider the last question about the on-chip just-in-time computation proposal as described.
After the raw photoreceptor data is read off the array, it can be channeled to multiple on-chip analog subcircuits for parallel processing.
Floating Gates for Mismatch Reduction
Retention Time: Benefits and Oxide Scaling
An obvious advantage of floating gate devices is the non-volatile nature of the charge storage. Indeed, 70-80˚A is the magic number for the oxide thickness below which floating gate retention times are too short for long-term non-volatile storage. The purpose of these transistors is to interface with off-chip devices that operate at higher voltages (such as 5V), but they can also be used for floating gate devices.
Any processes that allow these thicker oxide transistors will then support floating gate transistors with good retention times.
Calibration Complexity
- Calibrating the Calibration
- Continuous Adaptation
- Speed and External Algorithms
- Constant Programming Rate
- Art and Magic
A little extra complexity in calibrating a floating gate compared to other alternatives may be acceptable. However, the accuracy achieved by this method is limited by the matching of the capacitances on the floating gate. In practice, the operation of the floating gate device should be expected to change somewhat during a tunnel pulse.
30] developed a memory cell to solve the problem of the change in programming speed when the floating gate voltage changes.
High Voltages for Tunneling
Unfortunately, I have yet to hear of programs aimed at the more permanent component of an academic institution - the professors themselves. As another example, Caltech's GSC (Graduate Student Council) student newsletter, which is sent monthly to all graduate students, decided to run a multi-part series titled "The Graduate Student-Advisor Relationship." Thus, part of the initiative for change must come from those who do not personally have a problem.
This section provides a rough comparison of the difference in layout area, concluding that an equivalent digital circuit is about 30 to 100 times larger.
Mismatch in the difference circuit can be represented as offsets in the threshold voltages
Simulation results for programming the difference circuit to remove mismatch. The
The current-to-voltage circuit integrates current onto a capacitor for a prespecified time
Injection to the floating gate fgρ changes the offset of the difference current output. Left
Sweeping the floating gate voltage V f gγ changes the bias current I γ in the difference
Output of difference circuit to blank screen and edge stimuli
Output of difference circuit to corners
Block diagram of the sample-and-hold circuit
Transistor schematic of the sample-and-hold circuit
Generating a simultaneous pair of clock waveforms. Note that the circuit contains two
Channel charge in triode (top) and saturation (bottom)
