Interactive 3D: Using It, Designing It, and Teaching It
After several false starts, interactive,
real-time 3D graphics (IRT3D) is finally starting to happen. Advances in the
processing power and graphics capability of inexpensive desktop computers have
created the first generation of consumer-grade systems capable of generating
three-dimensional graphics of satisfying speed and quality. Now the user can
truly interact with these advanced representations in real time, creating powerful
new possibilities for the visualization of digital information.
While the first beneficiaries of this new capability have been those who play games on their computers, the potential of IRT3D to transform other aspects of human interaction with computers is being demonstrated more and more. From eCommerce to educational software, the rate at which IRT3D continues to improve is matched only by the demand of the market for real-time content. Media trends forecaster Jon Peddie & Associates estimates that by the year 2007 there will be more than one million 3D Web sites and over half a billion 3D-enabled Web browsers.
It isn't an overstatement to suggest that we may be watching an evolutionary leap in the progression of how computer images are created. And all of this is important to us as designers and educators because it not only presents us with exciting new methods to create expressions of information, but also with new skills that we must challenge our students to investigate and master.
Defining IRT3D
Unlike traditional sprite graphics, such as bitmapped images, 3D graphics are
based on mathematical descriptions of the surfaces of the objects that they
depict. Specifically, the surfaces are described in terms of polygonal geometry,
which "models" or approximates the surface as a blanket of polygons.
While bitmapped sprites can generally be rendered at a much faster rate, the
advantage of a geometric image is that the perspective from which it is viewed
can infinitely be changed.
Animation, on a computer or otherwise, consists of a series of slightly transformed images played in sequence fast enough that they appear to be a single, fluidly moving image. This is possible by virtue of a phenomenon called "persistence of vision" by which physiological components of the brain will "blur" successive images, so long as the images occur at or beyond the threshold rate of around eight frames per second (fps). In practice though, ideal results are realized at around 24 fps. Thus, 3D animation consists of a series of slightly transformed images, modeled from geometry, observed at a sufficient rate to look like real motion.
When the various images that make up an animated sequence are created in advance of their being viewed, they are said to be "pre-rendered." These often take the form of digital videos, which typically feature extreme detail.
Conversely, animated images rendered at the exact moment that they are viewed are called "real time." The trade-off of generating images in real time at rates sufficient for animation is generally the relative lack of detail that they can feature.
Animation that plays in a predefined sequence is said to be "linear." Pre-rendered 3D is generally linear. However, when we combine the ability of a computer to create 3D graphics in real time with its ability to be programmed, we have the ability to create 3D graphics that interact with the user in a responsive and logic-driven manner.
Using IRT3D
In looking at the actual use of IRT3D as a medium, there are six categories
that hold particular promise. It is particularly useful to bear these categories
and their distinctions in mind when beginning an exploration of IRT3D design
methods:
1. Games: This category consists of entertaining content that
involves the activity of play within a goal-based, competitive, and challenging
structure. IRT3D has been used in games for some time already, to great effect.
2. Non-Game Entertainment: Active or passive entertainment
experiences that are not goal-oriented. The most common example would be the
emerging medium of the "interactive narrative."
3. Informative: Activities that are driven by specific educational
goals. Content in this category is authored in a manner that teaches rather
than merely delivers knowledge. Examples are educational software lessons and
interactive instructions.
4. Reference: Content that is categorized or indexed in order
to accommodate its retrieval. Unlike "informative" content, reference
material is presented in whole based on query. Examples include maps, directories,
and simulations.
5. Persuasive: Content that seeks to influence the user, such
as to buy something or to assimilate a point of view. Examples include commercial
and political content.
6. Aesthetic: This category includes artistic or experimental
expression.
Most of my work with IRT3D in the past year has been to develop a formalized
approach to teaching it. There is little foundational pedagogy in this area
other than what can be synthesized from the various disciplines that it covers.
Still, I have pieced together and tested in the classroom "laboratory" what
seems to be a promising starting point. Using a series of key concepts as the
framework for a quarter-long (three-month) studio class, I have been able to
achieve significant results from my students (see "IRT3D: A Template Curriculum").
IRT3D: A Template Curriculum
Workflow and Tools
IRT3D authoring consists of two primary components: asset creation and
programming, in that order. The first step in comprehending the authoring
process is to understand this workflow, the various tools available, their
roles at each step, and the specific tools being used.
Modeling and Texturing for IRT3D
The overriding consideration in creating assets for IRT3D is to craft
them to accommodate the intense performance needs of real-time 3D rendering.
Key among these are low-polygon modeling techniques, as well as desktop
computer-friendly textures.
Transforms and Transformations
The ability to manipulate the position, orientation, and scale (PSR) of
an object in 3D space is the key to being able to program it to do practically
anything of use. The transform object is an abstraction of the complex
matrix math that remembers the PSR of a 3D object, while the transformations
are functions that allow the transform to be manipulated transparently.
Understanding what each is and how it works is the crucial first step
to learning how to program IRT3D.
2D to 3D Interface Approaches
Creating visual and conceptual metaphors for the user to interact with
objects in 3D space is a field of study in itself. We explore this in
terms of both visual and kinetic devices. Visual devices include the various
on-screen elements that the user can manipulate to affect the 3D world
and the objects in it, such as "dashboard" controls at the bottom of the
screen. Kinetic devices include those that involve user interaction with
hardware to affect the 3D world, such as mouse gesturing, picking, and
keystrokes.
Cameras and Perspective
This introduces the cinematic element. Aside from the technical chores
associated with creating and manipulating one or more cameras in 3D space,
there is now the design consideration of a psychological perspective.
Lighting
Also a cinematic element, but steeped in technical considerations. Lighting
is one of the most taxing features of real-time 3D rendering and must
be used judiciously. Still, it remains one of the most important and evocative
elements available in creating drama and mood, focusing attention, and
revealing/concealing geometry.
Groups and Hierarchies
Grouping allows multiple objects to be "joined" together to transform
as one, while hierarchies allow objects to take on a "parent-child" relationship.
Both groups and hierarchies have very profound influences on how objects
within them will behave. Used strategically, they can facilitate complex
control without the need for complex programming, as well as allowing
composite animation.
Effects
As more and more 3D functionality finds its way to the graphics processors
(GPU), more and more "effects" will become available in real time. Such
hardware-based effects offer a powerful means of achieving stunning and
seamless visuals without the need for a more powerful central processing
unit (CPU). Two common examples are particle systems and fogging, each
of which can be used to create dramatic and useful visual effects.
Designing IRT3D
It is just as important to have good authoring tools to create IRT3D content
as it is to have computers capable of delivering it. One of the primary reasons
IRT3D may well succeed now is that established companies have brought relatively
easy and broadly applicable IRT3D tools to market, all of which allow the authoring
of IRT3D content, using 3D content from existing tools.
In fact, the production of IRT3D taps the existing knowledgebase and functionality of established tools in each step of the workflow. For instance, a typical workflow might include creating 3D models and scenes in Alias|Wavefront Maya, texturing surfaces using Right Hemisphere's Deep Paint 3D, processing sounds and digital video in Adobe Systems' Premiere, and then authoring interactive 3D with these assets in Macromedia's Director Shockwave Studio.
Remember that the authoring tools at the end of the workflow are key to the process. Increasingly powerful, flexible, functional, multi-platform, object-oriented, and multi-output programs have now made a relatively easy task of what has been the
historically elusive task of creating viable IRT3D.
While this new breed of authoring tools has allowed designers to explore new ways of creating 3D visual
experiences that look and act more like real life, new design challenges do not end there. One particularly acute example is in the area of Human Computer Interface (HCI) for IRT3D. Traditional computer interface devices such as the mouse, track-pad, and screen are all 2D devices and fall somewhat short as intuitive and effective means of placing the hand of the user in the virtual 3D world of the computer.
While the initial work in IRT3D HCI research was focused on interfacing virtual 3D worlds with 2D devices, it has recently expanded to explore engaging more of the senses, so as to make the interface experience itself more natural and intuitive. One example is the work to create IRT3D interface devices that include haptic response, so that a user not only negotiates virtual 3D objects and space through the device, but feels them as well.
[Editor's note: Marty Plumbo will present a half-day workshop,
"The Design and Programming of Interactive 3D for Educators," on Sun., July
27, at Syllabus2003.]
