Securely Backing Up Syracuse University

• By Michael Kearns
• 09/01/04

In August of 2003, Syracuse University was hit with the MyDoom virus. During the downtime, hundreds of workstations re-booted themselves every ten minutes while administration spent time rebuilding and patching servers. Server outages can be catastrophic to the university, depending on which server is involved. For example, the lighting and temperatures in some labs are maintained via our energy servers. If this were stopped, the cost to the university and loss of reputation would be immense.

With the growing number of viruses proliferating across the Internet and affects of the MyDoom virus fresh in memory, Syracuse University started looking at a number of disaster recovery systems. They needed a system that could take a reliable snapshot of a server’s existing state at any given point in time without taking the server down. From that “hot” backup, the university needed to be able to quickly restore the server back to a clean, working state if the server ever experienced system corruption due to a virus attack or faulty system update. After examining the available options, Syracuse University chose the rapid, point-in-time restoration capabilities of V2i Protector from Symantec.

Syracuse was able to make real-time, disk-based backup and enable disaster recovery, as this system is designed to capture and encapsulate all files and configurations in one easy-to-manage backup file. Syracuse is now able to create full and incremental backups throughout the day without interrupting user productivity. It can restore failed or crippled systems back to an exact point in time without taking hours to rebuild, reconfigure, and restore systems from tape or other offline media.

Easing Recovery

Several factors have contributed to the effectiveness of V2i Protector in performing online backups at Syracuse—its minimal impact on server operations, speed at capturing data, and sector-based imaging that enables it to back up files while in use. For example, a SQL Server database is basically a huge open file. Using this Symantec system, the SQL Server at Syracuse University can be backed up and then restored to a completely different box. It comes up without any data loss, just as if it had been rebooted.

On a weekly basis, Syracuse uses V2i Protector to automatically perform full backups of the university servers. A schedule dictates when the backups run, what’s backed up, where the backup images should be stored, and then V2i Protector executes as directed, eliminating a lot of the manual processes associated with backups.

The time to restore a down server has decreased drastically. When restoring from tape-based backup, the process took from eight to 24 hours, not to mention the chance of running into unreadable tapes, corrupted files, or simply not finding the correct files. As a disk-based backup, Symantec’s backup system is inherently reliable and will also automatically send e-mails to notify users regarding the success of backup jobs.

Before implementing V2i Protector, the university was hesitant to apply several software security patches for fear of having to rebuild a server. Too often servers can’t handle a patch correctly or some other problem with the patch would occur, resulting in significant server downtime. With patches arriving on a weekly basis, the university couldn’t afford to keep exposing itself to the adverse side-effects.

The capabilities of the university’s new backup and recovery system were recently tested during a domain controller upgrade, one Friday afternoon, when the server suddenly became inoperable. Syracuse’s fallback plan simply takes a snapshot image of the operating system volume right before applying a patch. If the patch causes problems, the server can be rolled back to the exact state it was in before. Within 15 to 30 minutes, servers can be up and running.

Moving forward, Syracuse is building an off-site location for disaster recovery to allow the system to be more encompassing. It also plans to look at how to further integrate security and storage/backup systems to improve response times during virus outbreaks. The sooner a malicious threat can be identified by security technologies, the better the chance of making clean backups of mission-critical servers and patching the vulnerability without being exploited by the threat.