It Was the Best of Times, It Was the Worst of Times . . .
First, we had the budget cuts and were probably hitting this semester understaffed. Then, in many higher education-intensive parts of the country, the lights went out. Following that was the worm, Blaster. Then came, Sobig. And then the students came back, bringing with them their infected computers.
More than one CIO was quoted in the news media recently saying that this was the most difficult time ever to be IT staff on campus. And it was challenging, no doubt about it. But we think that, on campus after campus, the IT staff came through with shining colors. There was a lot to handle, but it was handled well!
So perhaps the most famous words of Dickens apply in all their discordant meanings:
"It was the best of times, it was the worst of times, it was the age of wisdom,
it was the age of foolishness, it was the epoch of belief, it was the epoch
of incredulity, it was the season of Light, it was the season of Darkness, it
was the spring of hope, it was the winter of despair, we had everything before
us, we had nothing before us, we were all going direct to Heaven, we were all
going direct the other way—in short, the period was so."
Impacts on the campus information technology infrastructure probably affected the campus-wide community, in general, as much as any one thing ever has. Read the following to yourself while imagining that you are listening to "Weekend Edition" on NPR and, as voices report from these campuses, there is stirring background music going on:
- At Auburn University lines of students wrapped around support offices two
weeks after the Blaster worm first struck, initially downing the residential
network.
- Temple University provided students with free anti-virus software, which
is probably why—along with 90,000 warning e-mails and 27,00 paper flyers—only
400 of 35,000 student computers appear to have been infected.
- Duke University was set up to filter out virus-laden e-mail (it filtered
out 2.5 million) and only a handful of machines on campus got infected.
- Harvard University puts incoming e-mail through a virus filter—in
its arts and sciences department, 36,000 infected messages were stopped in
the first 9 hours of the software’s implementation.
- Brown University’s network registration tool scanned newly-connected
machines and instructed the owners without needed patches and updates to go
get them—that was about half of all students.
- George Mason University cut off all residence hall access to 3,600 students
for a while—after many students failed to sign statements that they
had run anti-virus software and placed patches.
- At Columbia University a system-wide spam filter protected computers from
viruses, even so consultants were also busy working in the dorms with student
machines on site. Due to strong rules on computer user rights, however, student
computers at Columbia and Barnard were not quarantined from the network.
- At the University of Maryland, returning students going into the network
were directed to a Web site telling them to apply patches for the Blaster
worm; those who did not do so within a fixed period of time were kicked off
the network.
- At the University of Virginia, about 800 student-owned machines were kicked
out of the network by security "bots" and were not allowed back
in until obtaining CDs and loading up on protection.
- Oberlin College suffered "near meltdown" on August 21 due to
students returning to campus with infected computers—9 out of 10 Windows
machines were infected.
- University of North Texas was cleaning off 16 computers every hour and
a half—and charging students $30 to do it. Students were not permitted
to log into the network without first proving they had clean computers.
- Vanderbilt University shut down connections to 1,200 computers after finding
out that as many as one-fourth of all student computers were infected. It
took days to get service to them all turned back on.
- Salisbury University shut its residential network completely down for a
day, this after a 2-week period spent cleaning off 500 university computers.
- MIT shut off service to infected computers and blocks traffic to and from
suspected machines.
- At the University of Illinois, a team of 30+ network technicians worked
on students’ desks to patch and check student computers in the residence
halls. Some students faced a week’s delay in getting permission to get
connected.
Wow. And those scenarios were played out on several thousand campuses across
North America.
Now the analyses begin. How much did it cost us in overtime and lost productivity?
Yes, we really do need to take a look at our policies about the responsibilities
of users and how much support we will provide—and at what cost—to
student machines. On some campuses, networks were shut down for days. On others,
there was little impact. Why? In some cases it was due to long-standing procedures
and thoughtful implementation, ahead of time. On other campuses, fast-thinking
IT management whipped teams together and somehow managed to grab students’
cooperation and thinking to head off larger disasters.
Next week, we’ll have a guest opinion from J'e St. Sauver of the University
of Oregon, who’s been looking at what happened, where. He will share his
opinions about why some campuses got hit so hard and some didn’t—and
also share some lessons learned.
