Expanding HPC & Research Computing - The Sustainable Way
How one university is creating
cost-effective growth models
for data center and resource
expansion that are also
environmentally and
community-friendly.
WASTE HEAT from a rack
of Notre Dame's high-performance
computing
equipment provides
inexpensive wintertime
heat for a local public
arboretum.
INCREASED DEMANDS FOR RESEARCH and high-performance computing-- along
with growing expectations for cost and environmental savings-- are putting new strains on the campus data
center. More and more, CIOs like the University of Notre Dame's (IN) Gordon Wishon are seeking creative
ways to build more sustainable models for data center and resource expansion. Wishon is not only the CIO,
but he's also associate vice president and associate provost at Notre Dame, indicating the central role he plays
in supporting the data and resource needs of both administrative and educational programs, as well as, increasingly,
the university's strategic effort to grow the size and scale of its research initiatives.
Recently, CT spoke with Wishon about efforts underway at Notre Dame to respond to the upswing in HPC
and research computing at the university while reducing costs and the environmental impact of program growth.
Campus Technology: As CIO at Notre Dame, are you seeing increasing demands on your data center,
especially from HPC and research computing?
Gordon Wishon: The strategic objectives of the institution [to grow its research programs] are changing
the nature of the demands on the IT organization, and we are facing these challenges in an era of significant
economic challenges. At the same time, we're trying to focus on sustainability and preserving the
environment. This all places a significantly different and sometimes new set of challenges on the IT organization,
as it does at most educational institutions.
As our research program grows we are anticipating significant growth and demand for data center-provided
services. Even if we didn't have such a focus on expanding our research programs, we would continue
to see growth in that demand, driven by the need to continually update and improve the productivity of
our administrative processes and the services that we provide to our students and our faculty.
[Bottom line,] we are anticipating an increase in demand on our data centers. Handling the growth in a way
that is both cost-effective and sensitive to environmental concerns is front and center for us right now.
CT: Is IT support centralized at Notre
Dame?
Wishon: Historically we have been
much more centralized in our IT support
model and strategy than most research
universities, such as my previous institution,
Georgia Tech, where the responsibility
for IT service and support was
very highly decentralized. In recent
years there has been a growing trend [in
higher education] back toward centralization,
even at those institutions that
are highly decentralized. There's a
recognition now by the deans and the
heads of academic units of the true cost
of providing IT services for their faculty,
particularly services that are increasingly
required to be administered in a
more professional way so as to address
concerns about security and risk to the
institution.
At Notre Dame we are hopeful about
developing a support posture that effectively
captures the best of both centralized
and decentralized environments.
I'm currently collaborating with our
vice president for research, as well as
the deans, to determine the best support
model going forward.
CT: And how does server virtualization
fit into the picture?
Wishon: Virtualization, of course, is a
key technology that we are using to
address some of [our challenges].
We've done some work to estimate the
growth of demand for services, and the
growth of the number of server platforms
within our data center. Our projections
suggest that at the current rate
of growth over the past two years-- even
without factoring in a significant
increase in demand driven by research--
by 2010 we will see a growth in the
number of servers on our data center
floor from the current 558 to more than
700. If we were to meet all these growth
requirements with the former architectural
approach of standing up individual
servers and application suites to
support each new application, we
would have 700 individual platforms or
clusters of platforms, with the concomitant
increase in power consumption
and cooling requirements.
Two years ago, we began to experiment
with virtualization in our data
center, and in fact, of the 558 servers
that we have on our data center floor
today, over the course of the past two
years we've actually reduced the number
of physical servers to about 360. So,
today, the 558 servers are distributed
across the 360-or-so physical platforms,
which represents about a 34 percent virtualization
rate.
CT: What will you gain from doing that?
Wishon: What we're hoping to achieve
as we grow to the projected 700 servers
by 2010 is a 50 percent virtualization
rate. If we can get more aggressive with
our virtualization as we continue to
grow, we hope to meet this growth
demand of 700 servers without substantially
increasing the number of physical
platforms on the floor beyond the number
we have in place today.
That has some obvious effects on
reducing power consumption and the
demands for air handling and cooling.
If we can achieve a 50 percent rate of
virtualization by 2010, we will effectively
flatline our growth in consumption
of power and cooling, with the
effect of avoiding over a million dollars
in annual costs.
CT: So you will not only reduce growth
of the number of physical servers on
your data center floor through virtualization,
but you'll also achieve some
real net savings over time. Do you also
have plans for using cloud services
toward similar ends?
Wishon: In the longer term-- and I know
some institutions are already heading in
this direction-- rather than achieving our
net savings through virtualization, we'll
begin to promote and increasingly rely
upon cloud solutions, or if you will, outsourced
solutions that take advantage of
much greater economies of scale. I happened
to be talking to my counterpart at
Arizona State recently, Adrian Sannier,
and ASU is in fact hoping to reduce the
need for a data center, or perhaps eliminate
the need for a data center on campus
altogether, by leveraging the cloud
and cloud providers, especially for commodity
services such as cycles and storage.
Companies like Google, Amazon,
Microsoft, EVault, and others are able
to take advantage of economies of scale
and ultimately reduce the demand for
net power consumption and cooling
requirements.
CT: Are there other strategies similar to
virtualization that you are leveraging
within the data center?
Wishon: Another opportunity in the
data center is the use of technologies
such as Oracle RAC [Real Application
Clusters]. While Oracle RAC does not
initially reduce costs or environmental
impact, we believe that over 10 years it
will significantly reduce the cost of
ownership, thanks to reduced maintenance
costs and higher availability--
because it's a much more scalable and
manageable architecture. It also will
help us to avoid adding more platforms
in our storage environment, and in our
database environment. So that's very similar to the way virtualization is
allowing us to avoid adding more hardware
in the data center.
CT: What are some examples of things
you're doing to reduce environmental
impact?
Wishon: We've attracted public interest
about an effort where we're looking at
ways to recapture waste heat. Historically,
of course, the heat generated by
computing platforms and servers in our
data centers has just been exhausted to
the atmosphere, thrown away. As server
platforms are growing in use of blades
and tacking on more chips and processing
power into the same space, that's
generating not only increased power
consumption and cooling requirements,
it's also generating additional heat. If
we could find a way to capture waste
heat, perhaps recycle it in some way,
then once again we'd have a method of
reducing the environmental impact and
reducing our carbon footprint.
“If we achieve a 50 percent virtualization rate by
2010, we will effectively flatline our growth in
power and cooling consumption, avoiding over a
million dollars in annual costs.”
CT: So what are you doing to move in
this direction?
Wishon:We have a couple of successful
initiatives that have been coordinated
by and were the brainchild of our Chief
Technology Officer Dewitt Latimer and
a scientist in our Center for Research
Computing, Paul Brenner, who have
helped the city of South Bend, [IN], to
reduce some of its energy requirements
by recycling waste heat.
In the first example, the city runs an
arboretum/greenhouse facility that contains
a display of southwestern American
flora, that was a donation from a
biologist here at Notre Dame a number
of years ago. The winters in South Bend
are particularly harsh, and the city was
finding it increasingly difficult to heat
that greenhouse display facility through
the long winters, especially in the face
of rising energy costs. So, since we have
collaborated with the city to build out a
metropolitan area network, it turns out
that the arboretum facility sits within
close proximity of some fiber that connects
the university to parts of the city.
We were able to simply take a rack of
high-performance computing equipment,
particularly dense computers that
generated a lot of waste heat, and sit it
right down in the greenhouse facility. So
what was waste heat is now being used
to help heat that greenhouse facility.
That not only lowers the costs for the
city of South Bend to heat the greenhouse,
it also allows us to move that rack
off of our data center floor, so we don't
have the burden of cooling that particular
suite of equipment.
We're looking at other ways to leverage
this notion. We have another pilot
project now with the city of South Bend
and its sewage treatment facility, [also]
coordinated by Latimer. Sewage treatment
facilities try to raise the temperature
of solid waste material to a certain
point in order to generate bacterial
activity that breaks the waste down into
byproducts, one of which can be fertilizer.
So, rather than hauling the material
away to the landfill, the treatment
facility can generate a revenue source--
provided it is able to raise the temperatures
long enough and to the right levels
to produce useful fertilizer. So we are
engaged in an effort to capture waste
heat from the data center, direct it over
to the sewage treatment facility, and
help the city to solve the waste treatment
problem and offset some of its costs.
CT: It seems like IT's role is not only
changing within the university, it's
changing within the community as well.
Wishon: As we go forward, we're going
to focus on awareness and education as
well as our technology initiatives. I
think we can all do much better than we
have done in the past in addressing environmental
concerns and our impact on
the environment.
::WEBEXTRAS ::
Head to our Green Technologies solution center for the latest news, case
studies, research, features, and more.