3D graphics workstations (1984 at SGI – now)
• Display: real-time, pseudo-realistic images of 3D scenes
• Object and command specification: 2D, 3D and nD input devices (controlling 3+ degrees of
freedom) and force feedback haptic devices for point-and-click, widgets, and direct manipulation • Control over appearance: WYSIWYG (still WYSIAYG) • Application control: multi-tasking, networked
(client/server) computation and window management
• High-end PCs with hot graphics cards (nVidia
GeForce™, ATI Radeon™) have supplanted graphics workstations
• Such PCs are clustered together over high speed buses or LANs to provide “scalable graphics” to drive tiled PowerWalls, Caves, etc.
Comparison in Hardware
128KB RAM 2.0GB DDR2 SDRAM
400KB Floppy 500GB Hard Disk
9” Black & White 24” Color 512 x 342
68 dpi
1920 x 1200 100 dpi
Mouse Keyboard
Mouse Keyboard
Desktop WIMP Desktop WIMP
Classical time-sharing is dead, 1:n n:1
• PCs and Workstations merging in
distributed heterogeneous computer
networks (e.g., LANs, WANs, Internet and
clusters)
• But file-, print- and compute-servers
and network are still shared
•
Client/server
computing,
component
software technologies are dominant
paradigms
• NCs (Network Computers), thin clients
attached to powerful servers reprise
dumb terminals and provide central
control – haven’t really taken off
New Forms of Computing:
1990-2007
(1/9)
• Multimedia: text and graphics synchronized with sound and video
• Hypermedia: multimedia with hypertextual links (also called Interactive Multimedia)
• Discredited term: “Digital Convergence,” merging of digital television and distributed computing,
consumer electronics: set-top computers (e.g., for Interactive TV, Video-On-Demand)
• The Internet and Internet appliances
• Embedded computing (information appliances, Personal Digital Assistants)
• Ubiquitous/pervasive/ invisible/nomadic
computing, “active badges” a la Xerox PARC, with hundreds of devices per person (appliances, clothing, even body parts) ;
New Forms of Computing:
1990-2007
(2/9)
•
Virtual Reality:
CAVE™
fully immersive VR
New Forms of Computing:
1990-2007
(3/9)
New Forms of Computing:
1990-2007
(4/9)
New Forms of Computing:
1990-2007
(5/9)
New Forms of Computing:
1990-2007
(6/9)
Use feet for navigation,
freeing hands for other
uses
New Forms of Computing:
1990-2007
(7/9)
semi-immersive VR
Fishtank VR on a monitor
small volume, head-tracked interactive stereo
New Forms of Computing:
1990-2007
(8/9)
augmented VR
(via video see-through optics)
Video or optics superimposes
New Interaction technology
• Inexpensive interaction devices from research lab into marketplace : 2D and 3D graphics no longer “special”
• 3D (even time-varying, “4D”) interactive illustrations as clip art/clip models coming
New forms of user-interface (UI Lecture)
• 3D Widgets; gestures-based UI (Brown’s “Sketch”)
– VR demands new interaction technology
– tablet PCs and electronic whiteboards present new opportunities for pen-centric, gesture-based
computing
• Social interfaces (Microsoft’s Bob and Clippie bombed, other avatars may not)
• Agents for indirect control
New Forms of Computing:
1990-2007
(9/9)
• Kids using computer graphics in rides (e.g. Aladdin, Pirates of the Caribbean) and
gaming consoles (e.g. Nintendo Wii), with head-mounted displays and/or force-feedback input devices
Powerful, Inexpensive Processing
Chips are Key in graphics subsystems
• Advances driven by Moore’s Law
– price/performance improves 2x every 18 months due to doubling of number of transistors
– only exponential growth in technology except that of WWW
• CPU
– Newest processors are 64-bit, dual/quad/8 core
• Server: Dual-Core Intel Itanium 2™, Dual-Core Intel Xeon™, Dual-Core AMD Opteron™, Sun UltraSPARC T1™
• Desktop: Intel Core 2 Duo™, AMD Athlon64 X2™, IBM G5™, Mac ProTM Quad/8-Core
• Graphics subsystems (GPU)
– Commodity cards have taken over mainstream market (nVidia GeForce™, ATI Radeon™)
• Physics subsystems
– AGEIA PhysX PPU (Physics Processing Unit)
– Offloads physics processing from the processor and GPU
• Other subsystems on the way
Chip Technology Advances in Games &
Commodity Graphics Cards
• Last-generation game platforms and set-top boxes use high-end processors (128-bit architectures, great graphics capabilities)
– Nintendo GameCube™ (powered by ATI) – Sony Playstation 2™
– Microsoft XBoxTM
• Next-generation platforms promise unprecedented performance by utilizing multi-core processors
– Microsoft Xbox 360™ (powered by ATI) – Sony Playstation 3™
– Nintendo WiiTM
• ATI Radeon HD 2900 and nVidia GeForce 8800 Ultra are the top graphics cards in 2007 (approx. $500 - $600)
• Significant advances in commodity graphics chips every 6 months, outrunning CPU chip advances
– AMD Athlon 64 X2 processors have 243 million transistors – Radeon HD 2900 chip has 700 million!
• GPUs are now arguably more powerful than CPUs
and programmable CPU & OS losing their
hegemony as the GPU co-processor tail wags CPU dog! E.g., niches in scientific computing
• Workstation market (and vendors) bowing to
Application Distinctions
Two basic paradigms
• Sample-based graphics (left): discrete samples are used to describe visual information
– pixels can be created by digitizing images, using a sample-based “painting” program, etc.
– often some aspect of the physical world is sampled for visualization, e.g., temperature across the US
– example programs: Adobe Photoshop™, GIMP™ , Adobe AfterEffects™ (it came out of CS123/CS224!)
• Geometry-based graphics (right): (also called scalable vector graphics or object-oriented graphics) geometrical model is created, along with various appearance
attributes, and is then sampled for visualization (rendering a.k.a image synthesis);
– often some aspect of physical world is visually simulated, or “synthesized”
– examples of 2D apps: Adobe Illustrator™, Adobe Freehand™ (formerly by Macromedia), Corel CorelDRAW™
– examples of 3D apps: Autodesk’s AutoCAD2008™,
Sampled-based Graphics
(1/2)
• Images are made up of grid of discrete
pixels, for 2D “picture elements”
Mathematical
pixel grid LCD display
CRT beam illumination pattern
NB: Can’t resolve adjacent pixels on CRT
1 pixel
light intensity
• Pixels are point locations with
associated sample values, usually of
light intensities/colors, transparency,
and other control information.
They
Sampled-based Graphics
(2/2)
• Samples created directly in paint-type program,
or as sampling of continuous (analog) visual
materials. E.g., photograph can be sampled (light
intensity/color measured at regular intervals)
with many devices including:
– flatbed and drum scanners
– digital still and motion (video) cameras – add-on boards such as frame grabbers
• Sample values can also be input numerically
(e.g., with numbers from computed dataset)
• Once an image is defined as pixel-array, it can
be manipulated
– Image editing: changes made by user, such as cutting and pasting sections, brush-type tools, and processing selected areas
– Image processing: algorithmic operations that are performed on image (or pre-selected
Sampling an Image
• Lets do some sampling of CIT building
• A color value is measured at every grid point and used to color corresponding grid square
• Note: this poor sampling and image reconstruction method creates blocky image
3D scene
• Once image is defined in terms of colors at (x, y) locations on grid, can change image easily by altering location or color values
• E.g., if we reverse our mapping above and make 10 = white and 0 = black, the image would look like this:
• Pixel information from one image can be copied and pasted into another, replacing or combining with previously stored pixels
What’s the Disadvantage?
• WYSIAYG (What You See Is All You Get): No additional information
– no depth information
– can’t examine scene from different point of view – at most can play with the individual pixels or
groups of pixels to change colors, enhance contrast, find edges, etc.
• But recently, strong interest in image-based
rendering to fake 3D scenes and arbitrary camera positions. New images constructed by
interpolation, composition, warping and other operations.
Examples of 2D Image
Manipulation
(1/3)
• There are many things possible with sampled images: uses of computer imaging range from military
applications to entertainment, medicine, art, and design.
• The news: digitally “enhanced” images are more and more common. Usually just sharpening, color balancing, but sometimes much more. To “Photoshop something in”:
– Time Magazine’s O.J. Simpson cover (below left)
– Reuters’ doctored Beirut bombing photo (below right; note cloned smoke)
Examples of 2D Image
Manipulation
(2/3)
• People can now mutate into other people or objects through morphing, and can carry on conversations in different times and places
– Interactive Digital Photomontage, Siggraph 2004
• The belief in very strong connection between still or moving photorealistic images and reality is severed
– no way to tell if news photos are “real” photographs
– photographic evidence no longer considered
“proof” in court of law without clear provenance of the image
Examples of 2D Image
Manipulation
(3/3)
• In artwork the processes and techniques of
photography and painting are merging in the art of digital imaging
– Michele Turre: the artist, her daughter, and her mother, all at 3 years of age
• CS123 emphasizes geometry-based graphics but we do two assignments on sample-based graphics to learn simple image processing.
– image processing in general includes image
transformation: used for feature detection, pattern recognition, machine/computer vision, and most recently, for image-based rendering
– new field of "digital photography" (a.ka.
Geometry-Based Graphics
• Geometry-based graphics applications store mathematical descriptions, or “models,” of geometric elements (lines, polygons,
polyhedrons…) and associated attributes (e.g., color, material properties). Elements are primitive geometric shapes, primitives for short
• Images created as pixel arrays (via sampling of geometry) for viewing, but not stored as part of model. Images of many different views are
generated from same model
• Users cannot usually work directly with individual pixels in geometry-based programs; as user
manipulates geometric elements, program resamples and redisplays elements
What is Geometric Modeling?
What is a model?
• Captures salient features (data, behavior) of thing/phenomenon being modeled
– data includes geometry, appearance attributes… – note similarity to OOP notions
• Real: some geometry inherent
– physical (e.g., actual object such as a pump)
– non-physical (e.g., mathematical function, weather data)
• Abstract: no inherent geometry, but for visualization
– organizational (e.g., company org. chart) – quantitative (e.g., graph of stock market) • Modeling is coping with complexity
• Our Focus: modeling/viewing simple objects • Consider this:
Through 3D computer graphics, first time in human
history we have abstract, easily changeable 3D forms. This has revolutionized working process of many fields –
Lecture Topics
• We manipulated primitive shapes with geometric transformations (translation, rotation, scale). These transformations are essential for model organization, process of composing complex objects from simpler components.
• Hierarchical models and same geometric
transformations are also essential for animation
• Once object’s geometry is established, must be viewed on screen: map from 3D to 2D for viewing and from 2D to 3D for 2D input devices (e.g., the mouse or pen/stylus)
• While mapping from 3D to 2D, object (surface) material properties and lighting effects are used in
Decomposition of a Geometric
Model
• Divide and Conquer
• Hierarchy of geometrical components
• Reduction to primitives (e.g., spheres,
cubes, etc.)
• Simple vs. not-so-simple elements (nail
vs. screw)
Head
Shaft
Point
Hierarchical (Tree) Diagram of
Nail
• Object to be modeled is (visually) analyzed, and then decomposed into collections of primitive shapes.
• Tree diagram provides visual method of expressing “composed of” relationships of model
• Such diagrams are part of 3D program interfaces (e.g., 3D Studio MAX, Maya)
Composition of a Geometric
Model
• Primitives created in decomposition process
must be assembled to create final object.
Done with “affine transformations,” T, R, S (as
in above example).
• Other composition operators will be briefly
discussed later (e.g., Constructive Solid
Geometry (CSG) with Boolean operators).
Primitives in their own
modeling
coordinate system
Composition in world (root)
coordinate system Translate
Translate and Scale
What Kind of Math do We Need?
Cartesian Coordinates
• Typically modeling space is floating point,
screen space is integer
NB:Often, screen coordinates are measured top to bottom, based on raster scan
(0,0)
2D Primitives
Line
X Y
Polyline
X Y
Polygon
X Y
Circle
Curves
• Piecewise linear approximation
• Splines: higher-order polynomials
– piecewise curvilinear approximation
French Curves Draftman’s Spline
Mathematical Splines
Natural Cubic Spline:
Example 3D Primitives
Polyhedron
Sphere
Polyline
Patch
Conceptual Framework for
Interactive Graphics
• Graphics library/package is intermediary between application and display hardware (Graphics System) • Application program maps application objects to
views (images) of those objects by calling on graphics library
• User interaction results in modification of model and/or image
• Images are usually means to an end: synthesis, design, manufacturing, visualization,…
• This hardware and software framework is more than 4 decades old but is still useful, indeed dominant
Graphics Library
• Examples: OpenGL™, DirectX™, Windows
Presentation Foundation™ (no logo), X3D™
• Primitives
• Attributes
– color – line style
– material properties for 3D
• Lights
• Transformations
• Immediate mode vs. retained mode
– immediate mode: no stored representation, package holds only attribute state, and
application must completely draw each frame – retained mode: library compiles and displays
2D Hardware/Algorithms Outline
• Display Hardware: Raster scan vs. random
scan
• Drawing primitives by scan conversion
– Lines, polygons, circles and ellipses, characters, attributes (color, line style, fill pattern…)
• Clipping to clip rectangle: three
methods
– analytically compute intersections and draw clipped primitives
– test each pixel and write only if inside
– compute spans inside the primitive and fill entire span without testing
• Color Table
– indirect specification of (pseudo) color
– color correction, simple types of animation
Graphics Display Hardware
• Vector (calligraphic, stroke, random-scan) – still used in some plotters
• Raster (TV, bitmap, pixmap), used in displays and laser printers
Ideal Drawing Vector Drawing
Raster
Vector Architecture
• Vector display works with display list/file stored in refresh buffer
2D Raster Architecture
• Raster display stores bitmap/pixmap in refresh buffer, also known as bitmap, frame buffer; can be in separate hardware (VRAM) or in CPU’s main
memory (DRAM)
• Video controller draws all scan-lines at consistent > 60 Hz; separates update rate of the frame buffer
Drawing Lines
• For horizontal, vertical and diagonal lines all pixels lie on ideal line: special case
• For lines at arbitrary angle, pick pixels closest to ideal line (Bresenham’s midpoint “scan conversion” algorithm)
• For thick lines, use multiple pixels in each column or fill a rotated rectangle
Drawing Filled Polygons
1.
Find intersection of scanline with
polygon edges
3.
Sort intersections by increasing
x
5.
Fill the polygon between pairs of
Drawing Circles and Ellipses
circle outline
• Thick circle drawing by tracing
rectangular pen
• Treat such primitives as regions and fill
them, e.g., approximate them as
sequences of connected quadrilaterals
Drawing Characters
(1/2)
• For characters defined by small bitmap
– selectively write pixels of refresh
buffer corresponding to “true” bits
of character bitmap
• Transparent mode: don’t write 0’s – BitBlt
with OR (explained later)
• Opaque mode: write background color
for 0’s
• Descenders & proportional spacing easily
accommodated
Base
Drawing Characters
(2/2)
• Outline fonts: defined in terms of
(mathematical) drawing primitives (lines,
arcs, splines) and thus scalable, but more
CPU intensive (e.g. Adobe PostScript™,
Microsoft TrueType™)
• Font design (typography is highly skilled
specialty, involving graphical and
1-bit Bilevel Display
• Digital intensity value Digital to Analog Conversion (DAC) analog signal to drive electron beam
• Black & White (or any 2 colors, depending on monitor phosphor color)
• Original Mac resolution was 512x384 pixels, now from 640x480 up to 1600x1200, even 3Kx3K in IBM’s Roentgen AM-LCD display
• 1280x1024 memory – 15 nanosec/pixel => 32 pixel chunks stored in shift registers/buffers
Frame
increment Raster scan
generator
Pixel value
n-Bit Display
• 2
n
intensities or colors: 1
Image Display System
Look-up Table
• Any specific 2n colors may be inadequate (n typically
16-24 in low-end systems)
• Look-up table allows 2n colors out of 224 colors to
be used in one image, some other 2n in another
image
– 224 = approx. 16.7 million, exceeds eye’s ability to
discriminate (somewhere between 7-10 million) • Color table is resource managed (usually) by
Color Look – Up Table Operation
• Pixel value is indexed to color look up table (CLUT) where color is stored. Here we use only 12 bits (4bits per color) for clarity – typically, 24 bits are used
• CLUT look up done at video rates, overlapped with fetch and DAC!
• In 24-bit “true” color systems, 3 x 8 bits for R, G, B; each color has its own 8-bit CLUT (0-255)
• CLUT allows variety of effects
– pseudo coloring (LandSat images, stress diagrams, thermograms…)
– fast image changes: change table rather than stored image
BitBlt/RasterOp
(1/3)
• Logically operate on each pixel in
rectangular source and destination
regions in same or different pixmaps to
achieve dynamics, e.g., to move/scroll
windows on screen
• RasterOp (Source, Destination)
Destination
• In some implementations either S or D
may be masked, and need not be same
size
BitBlt/RasterOp
(2/3)
• AND (S,D): S can mask out pixels in D• OR (S,D): S is non-destructively “added” to D; used for painting, transparent and “kerned” characters (where characters extend beyond their boxes); not as useful in n-bit systems
Here’s how you can use them:
Let’s say you want to add some game sprites to background…
0 (black) AND anything is 0, 1 (white) AND anything is anything
AND
=
1. Mask out with black
OR
=
BitBlt/RasterOp
(3/3)
• Replace (S,D): S destructively replaces D, i.e., is deleted and copied on top of D (also called Move); used for making opaque characters, icons, scroll
• Copy (S,D) as above, but S is not deleted
• XOR (S,D) S selectively inverts D; used in 1-bit systems for rubber-banding/dragging, cheap cursors:
• Note: effects in color systems for all but replace may be weird
