Interactive 3D: Using It, Designing It, and Teaching It

• By Marty Plumbo
• 05/30/03

After several false starts, interactive, real-time 3D graphics (IRT3D) is finally starting to happen. Advances in the processing power and graphics capability of inexpensive desktop computers have created the first generation of consumer-grade systems capable of generating three-dimensional graphics of satisfying speed and quality. Now the user can truly interact with these advanced representations in real time, creating powerful new possibilities for the visualization of digital information.

While the first beneficiaries of this new capability have been those who play games on their computers, the potential of IRT3D to transform other aspects of human interaction with computers is being demonstrated more and more. From eCommerce to educational software, the rate at which IRT3D continues to improve is matched only by the demand of the market for real-time content. Media trends forecaster Jon Peddie & Associates estimates that by the year 2007 there will be more than one million 3D Web sites and over half a billion 3D-enabled Web browsers.

It isn't an overstatement to suggest that we may be watching an evolutionary leap in the progression of how computer images are created. And all of this is important to us as designers and educators because it not only presents us with exciting new methods to create expressions of information, but also with new skills that we must challenge our students to investigate and master.

Defining IRT3D
Unlike traditional sprite graphics, such as bitmapped images, 3D graphics are based on mathematical descriptions of the surfaces of the objects that they depict. Specifically, the surfaces are described in terms of polygonal geometry, which "models" or approximates the surface as a blanket of polygons. While bitmapped sprites can generally be rendered at a much faster rate, the advantage of a geometric image is that the perspective from which it is viewed can infinitely be changed.

Animation, on a computer or otherwise, consists of a series of slightly transformed images played in sequence fast enough that they appear to be a single, fluidly moving image. This is possible by virtue of a phenomenon called "persistence of vision" by which physiological components of the brain will "blur" successive images, so long as the images occur at or beyond the threshold rate of around eight frames per second (fps). In practice though, ideal results are realized at around 24 fps. Thus, 3D animation consists of a series of slightly transformed images, modeled from geometry, observed at a sufficient rate to look like real motion.

When the various images that make up an animated sequence are created in advance of their being viewed, they are said to be "pre-rendered." These often take the form of digital videos, which typically feature extreme detail.

Conversely, animated images rendered at the exact moment that they are viewed are called "real time." The trade-off of generating images in real time at rates sufficient for animation is generally the relative lack of detail that they can feature.

Animation that plays in a predefined sequence is said to be "linear." Pre-rendered 3D is generally linear. However, when we combine the ability of a computer to create 3D graphics in real time with its ability to be programmed, we have the ability to create 3D graphics that interact with the user in a responsive and logic-driven manner.

Using IRT3D
In looking at the actual use of IRT3D as a medium, there are six categories that hold particular promise. It is particularly useful to bear these categories and their distinctions in mind when beginning an exploration of IRT3D design methods:

1. Games: This category consists of entertaining content that involves the activity of play within a goal-based, competitive, and challenging structure. IRT3D has been used in games for some time already, to great effect.

2. Non-Game Entertainment: Active or passive entertainment experiences that are not goal-oriented. The most common example would be the emerging medium of the "interactive narrative."

3. Informative: Activities that are driven by specific educational goals. Content in this category is authored in a manner that teaches rather than merely delivers knowledge. Examples are educational software lessons and interactive instructions.

4. Reference: Content that is categorized or indexed in order to accommodate its retrieval. Unlike "informative" content, reference material is presented in whole based on query. Examples include maps, directories, and simulations.

5. Persuasive: Content that seeks to influence the user, such as to buy something or to assimilate a point of view. Examples include commercial and political content.

6. Aesthetic: This category includes artistic or experimental expression.

Most of my work with IRT3D in the past year has been to develop a formalized approach to teaching it. There is little foundational pedagogy in this area other than what can be synthesized from the various disciplines that it covers. Still, I have pieced together and tested in the classroom "laboratory" what seems to be a promising starting point. Using a series of key concepts as the framework for a quarter-long (three-month) studio class, I have been able to achieve significant results from my students (see "IRT3D: A Template Curriculum").

Designing IRT3D
It is just as important to have good authoring tools to create IRT3D content as it is to have computers capable of delivering it. One of the primary reasons IRT3D may well succeed now is that established companies have brought relatively easy and broadly applicable IRT3D tools to market, all of which allow the authoring of IRT3D content, using 3D content from existing tools.

In fact, the production of IRT3D taps the existing knowledgebase and functionality of established tools in each step of the workflow. For instance, a typical workflow might include creating 3D models and scenes in Alias|Wavefront Maya, texturing surfaces using Right Hemisphere's Deep Paint 3D, processing sounds and digital video in Adobe Systems' Premiere, and then authoring interactive 3D with these assets in Macromedia's Director Shockwave Studio.

Remember that the authoring tools at the end of the workflow are key to the process. Increasingly powerful, flexible, functional, multi-platform, object-oriented, and multi-output programs have now made a relatively easy task of what has been the historically elusive task of creating viable IRT3D.

While this new breed of authoring tools has allowed designers to explore new ways of creating 3D visual experiences that look and act more like real life, new design challenges do not end there. One particularly acute example is in the area of Human Computer Interface (HCI) for IRT3D. Traditional computer interface devices such as the mouse, track-pad, and screen are all 2D devices and fall somewhat short as intuitive and effective means of placing the hand of the user in the virtual 3D world of the computer.

While the initial work in IRT3D HCI research was focused on interfacing virtual 3D worlds with 2D devices, it has recently expanded to explore engaging more of the senses, so as to make the interface experience itself more natural and intuitive. One example is the work to create IRT3D interface devices that include haptic response, so that a user not only negotiates virtual 3D objects and space through the device, but feels them as well.

[Editor's note: Marty Plumbo will present a half-day workshop, "The Design and Programming of Interactive 3D for Educators," on Sun., July 27, at Syllabus2003.]