2006 Campus Technology Innovators: QA
TECHNOLOGY AREA: QA
Innovator: Wayne State University
CAMP: WSU’S quality-assurance processes
are saving the school hundreds of thousands
of dollars a year.
Challenge Met
Today’s students and faculty routinely expect
to access integrated online systems via the
web 24/7; such self-service systems have
become a competitive differentiator. Realizing
that as early as seven years ago, Wayne
State University’s (MI) CIO John Camp
began working toward cost-effective methods
to consistently ensure the overall availability,
performance, function, and integrity of the
university’s major applications.
“Most colleges and universities today
don’t adequately test changes to their systems
before they are implemented,” Camp
says. To address that, WSU wanted to reduce
money and time spent testing software, and
to improve end-to-end testing of major applications.
In addition, once systems moved
into production, Camp and his team wanted
the ability to monitor them remotely.
WSU (which serves 33,000 students in
11 schools and colleges as Michigan’s only
urban research university) now saves hundreds
of thousands of dollars a year through
an ever-growing library of sophisticated,
automated quality-assurance software and
practices. For example, the university estimates
it used to spend about $275,000 a
year testing one major software release,
along with other minor releases. Automated
testing saves nearly 85 percent of that cost.
The university’s holistic approach to testing,
highly unusual in higher ed, involves both
pre-production load, functional and integrity
testing, and pre- and post-production service
monitoring from customer locations.
How They Did It
To meet the challenge of improving testing
systems while reducing time and cost, Camp
and his team selected quality-assurance and
-testing products from Compuware. Using those products, QA
staff document all major functions of each
application, identify the mix of transactions to
validate performance, and write reliable,
repeatable automated scripts that provide
regression results for analysis and testing.
The university’s QA department also has
developed procedures to identify and
document functional processes in WSU’s
core applications, which include SunGard’s
Banner and Luminis suites, Cognos’ reporting environment, and
Blackboard’s Academic Suite.
In order to simulate transactions within
the university’s core applications and report
back on problems, WSU deploys software
“robots” (from Compuware’s Vantage products) in four strategic locations. The robots
monitor network traffic, application use, and
latency 24/7, and alert IT support personnel
if services aren’t available or performance
lags.
For further performance monitoring, the
university employs 70 counters to monitor
each of the 100-plus data center servers
(Unix and Windows) for tasks such as file
space, paging, cache, and CPU utilization.The
Vantage software issues alerts when any of
the counters exceed acceptable thresholds.
Along with Compuware, Camp says, both
Sun Microsystems and SunGard Higher Education
were important partners in the project.
Because Sun equipment runs many of WSU’s
enterprise applications, WSU and Sun
worked together to design, test, and optimize
how SunGard applications run on Sun’s server
and storage products.
Next Steps
Since there are thousands of processes to
be tested across all of the university’s applications,
developing a library of test scripts
for each application is a multi-year project.
For now, WSU plans to: complete test documentation
and scripting for all of its Sun-
Gard and Luminis suites and integrated
applications, including Blackboard; expand
performance monitoring on the school’s
web pages, including the various hubs that
interconnect the campus and its extension
centers; and extend its QA approach to
other areas, such as incident and problem
handling, change management, and disaster
recovery.
WSU is also developing a business plan
for a non-profit corporation (owned by colleges
and universities) that will provide a
testing service for members.
Advice
Choosing the right vendor partner is critical.
Camp says that WSU’s selection of Compuware
as a tools vendor brought the support
and commitment needed in numerous
ways, including initial product training,
numerous no-cost “lunch and learn” sessions
for staff, help with coding, review of
testing plans, and excellent customer service
overall.