Click here to receive your FREE subscription to Campus Technology
Current News
Sharing Large Data Files
10/29/2003
J'e's back! While away, he had an aaaHHHAAA!!! moment, similar to his first browsing experience, and thinks he may just have spotted the "Internet2 Killer App." Read on to see why.
------------------------------------------
by J'e St. Sauver
University of Oregon Computing Center
This year's Fall
Internet2 Member Meeting took place in Indianapolis, from October 12th to
17th. Besides being a nice opportunity to learn what's been going on in the
Internet2 community (while also providing a chance to hash out issues with colleagues
from other I2 schools face-to-face over a beer and a bowl of Cincinnati-style
3-, 4-, or 5-way chili), the I2 Member Meeting included the widely overlooked
announcement of what may very well be the long-awaited "Internet2 killer
app," a program from the University of Tennessee Knoxville Computer Science
Department that g'es by the somewhat odd name of LoRS, part of the LoCI
project.
Watching the LoRS demo at the Indy I2 meeting gave me the same sort of "aaaHHHAAA!!!" moment that I recall from when I first saw someone use an early version of Netscape to access a simple Web page: clearly, here was something that's going to profoundly change the way we do things online.
If you happen to have attended the LoRS session like I did, then you had had the chance to see an application that satisfies a fundamental need, much in the way that e-mail or the Web d'es. The need that LoRS satisfies is the need to be able to efficiently distribute large files, files that are too big to conveniently send by e-mail, files too big to conveniently download via the Web. You know the sort of files I'm talking about - large multi-gigabyte (or even multi-terabyte) experimental physics datasets, or CD-sized Linux ISOs, or those wonderful multi-hundred megabyte PowerPoint marketing presentations we all so love.
Yes, I know: people currently do move large (if not huge) files all the time via ftp, or chopped into digestible chunks via e-mail, or via Web pages.
Unfortunately, when folks move files using traditional tools, they don't tend to get very good network throughput, even over well-engineered, high-capacity, lightly loaded networks like Internet2's Abilene. (For example, the median throughput on Abilene for bulk file transfers is still less than 2.5Mbps. See Table 1 in the I2 weekly NetFlow report . One reason you're not seeing experimental physicists with fast Ethernet connections routinely saturating 100Mbps links is no mystery: it is simply a manifestation of our old friend, the TCP bandwidth delay product and its negative impact on untuned single-threaded network application throughput. (For a nice discussion of this, see: http://www.psc.edu/networking/perf_tune.html).
Serving large files from a single location, or even from a comparatively small set of distributed mirrors, also d'esn't scale very well. Ask anyone who hosts a Linux distribution mirror what they see when a Linux distributor kicks a new release out the door!
Recommended Reading
- News Update :: Tuesday, November 25, 2008
:::::: NEWS
: Institute for Cyber Security at U Texas, San Antonio Opens Incubator
: ISO/IEC Publishes Office Open XML Standard
: Dynamics NAV 2009 ERP Coming Next Month
: Southwest Baptist U Adopts Angel LMS
: IE8 Release Candidate Coming on Q1 2009
: Hodges U Credits Lecture Capture for Online Enrollment Boost
: Mitsubishi Debuts WXGA Projectors for Education
: Slippery Rock U Applies SAS to Data Management and Reporting
: Moodle Gets SCORM Improvements, Security Fixes - IT Trends :: Thursday, November 20, 2008
:::::: CASE STUDY
:: DePaul Sets the Bar in Student Relationship Management
:::::: IT NEWS
:: Microsoft Unveils Exchange and SharePoint as Services
:: Penn State Pilots Proctored Online Testing System from Kryterion
:: State-wide New Mexico E-learning Program Adopts Wimba for Collaboration
:: IBM Launches 'Pass It Along' Social Networking, Knowledge Sharing Tool
:: MIR3 Adds Recorded Response Feature to Mass Notification Service
:: Northern New Mexico College Moves to Latest AVG Anti-Virus
:: Ubuntu ARMed for Mobile Expansion - SmartClassroom :: Wednesday, November 19, 2008
:::::: SPOTLIGHT
: 6 Ways Not To Become Rote Using Instructional Technology
:::::: NEWS and PRODUCT UPDATES
: CSU System Adopts Moodle LMS Services
: CourseCast Integrates Closed-Captioning Service
: Wimba Pronto 2.1 Adds Accessibility, Spanish Language Features
: Mitsubishi Debuts XD95U 'Pico' Projector
: InFocus Launches DisplayLink Projector for Education - Web 2.0 :: Wednesday, November 19, 2008
:::::: THE BUZZ
: Bringing Student Web "Stuff" to Campus Enterprise Systems
:::::: WEIGHING IN
: Tipping Point for "Content"--Dynamic Interaction, Not Static Stuff
:::::: PRODUCTS AND APPS
: Delta iTunes U Helps Meet Student Expectations for Web 2.0 Apps
: Penn State Pilots Proctored Online Testing System from Kryterion
: State-wide New Mexico E-learning Program Adopts Wimba for Collaboration
: CSU System Adopts Moodle LMS Services
: Office Web Apps Will Work on iPhones - News Update :: Tuesday, November 18, 2008
:::::: NEWS
: Carnegie Mellon Validates Production Xirrus 802.11n Network
: Ave Maria U Deploys In-Building Cellular Gear
: ASU Campus Nixes Fiber; Chooses Gigabit Wireless for Network Connectivity
: Sun Unveils Family of Open Storage Appliances
: Office Web Apps Will Work on iPhones
: Sun To Cut More Than 15 Percent of Global Workforce
: Texas A&M Upgrades Supply Chain Lab Curriculum
: Texas Lutheran U Implementing Jenzabar EX for ERP
: Anna U Chennai Partners with Cypress on Embedded Systems Engineering - Campus Security :: November 14, 2008
:::::: SECURITY SPOTLIGHT
: Smart Phone Security: New Challenges for Road Warriors
:::::: CAMPUS SECURITY NEWS
: SMobile Releases Antivirus To Protect Google Android Phones
: Blue Coat Integrates Network Appliances
: e2Campus Provides Twitter Integration in Emergency Notification System
: Moodle Gets Student Verification Capabilities
: Rave Wireless Adding BlackBerry Devices to Notification Service
: U Miami Trades IPS for Top Layer Security System
: Cornell Hardens Campus Network with Gigabit Wireless Radio Links
: U Pittsburgh Turns to Verizon Business for Automated Notification Services
