As technology becomes integral to teaching and learning, more instructors engage in technology development and digital scholarship. How d'es an institution measure and evaluate faculty work in instructional technology? What should be the reward for leading the way in uncharted waters? Our Syllabus Institute participants discuss the issues.

Last spring, Leslie Whitman was denied tenure at the University of Indiana. Whitman's scholarship contributions included one article that had been published and one that had been accepted, but not yet published, in printed, peer-reviewed journals. Two other articles had been published online. Whitman had also created one Web-based course and was planning a second online course. Although Whitman was acknowledged as developing and introducing new curricula, among other criticisms of the bid for tenure wasthe difficulty assessing the quality and impact of Whitman's work, and its significance to the overall body of knowledge.

Leslie Whitman's bid for tenure may have been denied based on a relatively light publishing and teaching record. But as a fictitious subject of a case study, Whitman became an example of the difficulty in measuring and evaluating faculty work in digital scholarship. Discussed at a forum on faculty rewards in the digital age at the University of Indiana last spring, Whitman's case pointed out that both faculty and administrators are skeptical about the efficacy of online teaching and research.

As technology becomes more immersed in the teaching and learning process, however, the need to develop processes and standards for online scholarship--and its accompanying reward structure--becomes more critical. In July 2000, the second annual Syllabus Institute was held at Stanford University. The purpose of the Institute is to create opportunities for exchange and reflection among leading educators, practitioners of educational computing and advanced interactive technologies, with a goal of influencing the course of information technology in higher education.

The Institute hosted five tracks, two of which addressed issues related to "Curricular Change and Development" and "Faculty Rewards and Evolving Roles." These two tracks met as a single group, and over the one-and-one-half-day meeting, four areas emerged as having direct bearing on curricular change and faculty rewards in digital instructional environments: defining rewards, institutional cultures, collaborative commercial and peer support, and tenure and promotion review procedures. This article summarizes the group discussion of these topics.

Defining Rewards

Some of the group felt that tenure, per se, should not be viewed as a reward. The sense was that awarding tenure based on teaching excellence alone would fit few institutional cultures. Thus, rewards were defined as raises, promotions (other than tenure), bonuses, travel and research stipends, office and lab space, lab equipment, reduced teaching loads, and support of an environment for enhancing technology-based instruction.

Institutional Cultures

In "Faculty Engagement and Support in the New Learning Environment," one of the group participants, Paul Hagner, addresses the cultural aspect--among others--of teaching and reward transformation in digital environments. According to Hagner, institutional cultures might be viewed as a matrix of high/low trust and high/low openness to innovation across high/low resource environments. He argues that various faculty engagement options might be considered, based on an institution's

"cultural mix," derived from the matrix. Thus, it is important for faculty to know the culture of their institution and to fit their vision of technology to that culture. Violating this principle can result in disaster, or at least severe consequences regarding tenure and promotion, particularly for young faculty. As one member said, "How can we, with a clear conscience, recommend that new faculty develop pedagogical approaches using technology, with traditional tenure and promotion procedures in place?"

However, the group also noted that in teaching institutions (versus research-intensive institutions), these cultural or traditional conflicts do not enter the equation much. Change occurs in the curriculum from theory to practice by using case studies and learning by doing, with an emphasis on developing skill sets for students.

A less complex view of an institution's culture was defined by one member of the group as a conglomerate of categories made up of faculty and administrators who are:

• Information technology-aware
• Intellectuals--they always want to see how an idea will work before implementing it
• Accountant types--they always want to know the bottom line first
• Those who should either retire, leave academia, or accept change.

Collaborative, Commercial, and Peer Support

Collaborative efforts involving the commercial sector and students and faculty from other departments are very important for faculty engagement, curricular development and change, and technology integration. One member described an institution's approach to the dilemma of too many students for the number of faculty available to teach business courses:

• The undergraduate curriculum is being redesigned to improve efficiency by increasing online delivery of curricula.
• Philanthropy has provided resources for faculty release time for those developing the new curriculum.
• The savings in costs are being passed on to the support of graduate business education.
• "What every manager needs to know" is the philosophy being used in the redesign, with the help of a technology expert to guide the development of the undergraduate framework.
• In the redesign, it was recognized that technology, ethics, and team building are skills that must cross the entire curriculum, but sometimes overcoming departmental lines is difficult. In some cases, it takes a year to iron out a new curriculum.

Another member described working on an integrated engineering curriculum through a "virtual enterprise" approach. Information technologies are the most highly rated tools for engineers and are second only to design as the most commonly used tools. For accreditation, students are required to form connections with alumni and employers, who determine measures for the success of their enterprise. Information technology is the most recognized practice by these external groups. At this particular institution, engineering education lacked practice in information technology, so a "teaching factory" was developed--just as physicians have a teaching hospital--to build better practitioners in engineering. The "factories" allow students to work on industry-sponsored projects, becoming a virtual enterprise of a full-scale product manufacturing and supply chain.

Tenure and Promotion Review Procedures

Will it be possible for an institution's administration to acknowledge that a structure should be put into place to reward innovative use of instructional technologies? The Syllabus Institute considered several ideas and approaches.

First, a national effort should be made for tenure, promotion, and merit recognition, including a Web site with links to institutions that have plans and/or guidelines for the review of technology-enhanced instruction.

In addition, recognition and support of Web sites would provide peer review of instructional technology efforts. Several institutions have already implemented different interpretations of this idea.

• MERLOT, Multimedia Educational Resource for Learning and Online Teaching, is a site where visitors can locate learning materials within their disciplines as well as colleagues who share their discipline/interests (see www. merlot.org, and Syllabus, v14, no. 3, p16, October 2000).
• The University of Indiana at South Bend's online journal,The Journal of Scholarship and Teaching, includes reports on teaching and learning, formal research articles, classroom action research, and essays on the scholarship of teaching and learning (see www.iusb.edu/~josotl).
• The Technology Source, a peer-reviewed bimonthly online periodical supported by the University of North Carolina, provides thoughtful, illuminating articles to assist educators as they face the challenge of integrating information technology tools into teaching and also into managing educational organizations (see http:// horizon. unc.edu/ts/).

Another component to the solution is implementation of information technology promotion, tenure, and merit guidelines. An action team appointed by an associate provost at Southeast Missouri State University has already developed a rationale and set of guidelines for evaluating the use of instructional technology (http:// cstl.semo.edu/itfrr). Evaluation criteria are centered on contributions to scholarship, teaching, and service. A list of examples of the uses of technology for each criterion is given, and includes in part:

Scholarship

Publication in online journals

Conference presentations

Development and dissemination of application software in the discipline<>

Teaching

Online courses

Use of computers in the classroom

Assignments requiring use of instructional technology

Use of course or discipline-specific software

Course Web page development

Service

Informal technical assistance to colleagues

Workshops for community members or colleagues

Technical consultation

Service on technology-oriented committees

Another proposed solution is to create a "council" for an elective review and feedback process in which peers carry out expert review of faculty innovations in technological products and processes. The council would have specific features, including:

• Dual functionality as an elective mechanism for the external evaluation of promotion and tenure information and a critical review process for new technological products and processes.
• Council reviews that are not discipline-specific; they would apply equally well to technology products and processes related to learning, research, and service.
• Council-specified evaluation procedures and criteria to be used in judging technological products and processes that faculty choose to present as part of their dossiers/activities.
• Local Tenure-Initiating Units (TIUs) to provide written guidelines regarding quality standards germane to reviews within their disciplines.

Recommendations and Unanswered Questions

To assess contributions to the body of knowledge in an academic field of computer-disseminated research, institutions need to identify and develop specific strategies and objectives. In addition, as part of the tenure and promotion process, rewards need to be put in place for faculty engaged in new research-related uses of computer technology. Finally, research on the impact of new computer technologies on the discipline is an important area of scholarship, and institutions need to encourage and reward faculty doing this kind of work.

At the close of the Syllabus Institute, the exchange of ideas and collaborative reflection had produced a series of recommendations, but had also raised a host of questions that have yet to be answered:

• How can we change the system to facilitate and reward innovative pedagogy using technology?
• What curriculum changes can we or should we make to ensure that our students are prepared to be technologically competent?
• What value should be placed on a strong technology background in hiring new faculty?

Tenure and promotion are based on a faculty member's contribution to scholarship, teaching, and service. In research-intensive institutions at least, and for the foreseeable future, this will be interpreted as publish, publish, and publish.