Internet2 Kicks Off 2025 with a Major Cloud Scorecard Update
A Conversation with Sean O'Brien and Loren Malm
Internet2 has maintained its Cloud Scorecard service since the creation of a pilot in 2021, helping the research and higher education community weigh tough decisions as they learn about, compare, and select cloud solutions.
With the proliferation of applications and providers ready to compete in today's burgeoning cloud marketplace — completed Cloud Scorecards now number into the 80s — it's the perfect time for a relaunch that will find users examining the complex issues and ever-growing importance of the modern cloud.
A public preview of the latest release on the Internet2 Cloud Scorecard Finder website was made this past December [2024], launching new features that include dynamic selection criteria and options to explore multiple solutions side-by-side with easy-to-interpret views. Further enhancements are set to be added in the new year, expanding the platform's capabilities.
The development work and advances in the updated Cloud Scorecard offering reflect the community-based engagement Internet2 is known for. Here, CT gets together with Internet2 Associate VP Sean O'Brien, who leads Internet2's NET+ and cloud services programs, and Ball State University VP for IT and CIO Loren Malm, who serves on the Cloud Scorecard advisory committee. We ask them about the new Cloud Scorecard release and the development efforts behind it.
Mary Grush: There's a lot of "buzz" around the Cloud Scorecard updates and the new Cloud Scorecard Finder website. Sean, for some perspective, what was Internet2's original reason for building a cloud scorecard?
Sean O'Brien: During the early part of this decade, there was amazing growth in the utilization of cloud services at U.S. research and higher education (R&E) institutions. At Internet2, we wanted to leverage some of the cloud evaluation criteria that we look at for NET+ cloud services, and build on that. We wanted a way to scale some of the due diligence that NET+ does for select cloud solutions. So, we got a working group together to think about the two-dozen-or-so key questions we would want to ask every cloud service vendor who might be interested in working in the higher education market.
We wanted a place for higher education institutions to find the most important things they need to know about a cloud service — things like how it meets various compliance or regulatory standards, or how it integrates with existing services on a campus environment. That's where we started, back in 2021.
We wanted a place for higher education institutions to find the most important things they need to know about a cloud service.
Grush: Loren, what are some of the challenges higher education IT leaders face, finding and evaluating cloud solutions? How can the Cloud Scorecard Finder help?
Loren Malm: I think one of the drivers that has made the Cloud Scorecard Finder so appealing at higher education institutions for people in positions like mine — for CIOs — is that it's a way to work more easily with the dramatic increase in the number of vendors that are available to choose from. We need to understand what capabilities cloud services offer, as well as what their limitations are. The Cloud Scorecard Finder brings all the key information — particularly based on factors that we've identified as being the most important — to one place where campus technology leadership can, relatively quickly, do some sorting and identification of which vendors might be the most relevant for the things they are looking for.
Campus technology leadership can, relatively quickly, do some sorting and identification of which vendors might be the most relevant for the things they are looking for.
At my own institution, the Cloud Scorecard is an integral part of our workflow when looking for new solutions. It's one of the first resources we turn to, putting us not only on a good track to find solutions more quickly but also to discover solutions we might not otherwise have considered.
Grush: How is this new version of the Cloud Scorecard service different from the original that we've had since the 2021 pilot?
O'Brien: Originally we just wanted a place for people
to be able to find vendors' completed questionnaires. So we launched what was essentially a Wiki site — all plain text. There was nothing very dynamic about it. But there were a couple good reasons for the simplicity of the original version. First, it was all about getting something out there. Second, before investing community resources into building something more, we wanted to make sure the higher education R&E community is actually interested in it — to ensure that it will provide value, and to be confident that vendors will participate and fill out the questionnaires.
In the time since the 2021 pilot, we've reached a new level of confidence and proof-of-concept. We've reached consensus that that yes, the higher education R&E community is interested; yes, cloud vendors will fill out the questionnaire; and yes, cloud decision makers will use it. That's what led us to launching the new Cloud Scorecard Finder site you can see today.
We've reached a new level of confidence and proof-of-concept. That's what led us to launching the new Cloud Scorecard Finder site you can see today.
There was a significant selection process related to its development criteria, but ultimately what we wanted the Cloud Scorecard Finder site to do was basically three things: One, to improve the overall user experience — to have a friendlier, more intuitive, more modern user experience. Two, to increase the ability for higher education institutions to find or discover a cloud service that meets their needs. And three, perhaps the biggest advantage or change, to power the Finder site with the ability to sort based on compliance requirements, on integration, and on type of services. So now, we have dynamic filtering and sorting in place on the new Finder site that will help institutions identify which services meet the specific needs that they have. There's a stark difference between what we have now and the modest Wiki site we had originally. Now we have a full feature set that we know delivers value to the higher education R&E community.
Now we have a full feature set that we know delivers value to the higher education R&E community.
Grush: It sounds like the new Cloud Scorecard Finder is miles ahead of the original version, especially with the dynamic sorting capabilities.
O'Brien: Yes, I think the biggest step forward to date is the ability to do the dynamic sorting and filtering that I've mentioned.
A CIO has the ability to come to the site with a cloud services challenge or an opportunity that they're trying to address, and after some input criteria for factors like security or accessibility, they'll see a dynamic list of vendors who self-attest to meeting their requirements.
The Cloud Scorecard Finder is not merely a listing where people can find names of products they already know about or might be hearing about, but a pathway to discovering solutions that meet their needs — solutions that an organization may not even know about yet.
Grush: It sounds like with the dynamic sorting function of the Cloud Scorecard Finder, you have a strong education or informational element that comes along with that.
Malm: That's right. One example I like to give is something that has come up recently, relating to AI. The Cloud Scorecard committee is continuously reviewing the questions we're asking cloud solutions vendors. And we work with other groups within Internet2 as well — for example, there's a business process and legal group that we routinely talk with in the context of developing the questionnaire.
Because of our ongoing oversight and refinement of the questionnaire, along with input from other Internet2 groups, one of the things we knew we needed to ask about in the questionnaire, was AI. We needed a question about AI.
People don't always think about AI — even though AI is everywhere, they don't always focus on it with every solution they're looking at. When I look for a solution in a given space, I might not be thinking about the fact that it's using AI in the background. But a comment that a vendor might make as part of the Cloud Scorecard, could prompt me to think about AI even if I wasn't considering it going in. I think that's the kind of education or discovery process you're asking about.
Grush: Yes, exactly. What is the makeup of the current Cloud Scorecard advisory committee, and what is that group's ongoing responsibility to oversee the Cloud Scorecard?
Malm: One of the important characteristics of the original working group (that's now morphed into an ongoing committee), is that it brings together a highly diverse set of stakeholders to make sure that the questions we build into the questionnaire reflect, to the best of our ability, and encompass all of the things that different campus stakeholders might care about.
So if you look at the current composition of the committee, we have a CIO. We have a director of a university contracting office. We have a chief information security officer. We have the vendors themselves in some disciplines. We tried to cast a wide net for the committee. This ensures that when we're putting together a set of questions, while we try to confine those questions and make it a relatively lightweight task for vendors to fill out the questionnaire, it still truly is representative of what diverse stakeholders would be thinking about across the research and academic enterprises within an institution.
Grush: How did the committee shape the direction for the new Cloud Scorecard Finder?
O'Brien: The advisory committee was instrumental in ensuring that Internet2 — and our members more broadly — maintain control over the Cloud Scorecard's destiny. The new Finder is built on an open source Drupal platform developed by Cornell, offering us the flexibility and autonomy needed to manage the data effectively. Leveraging this open source platform aligns with Internet2's core ethos: investing in community-driven projects that deliver shared benefits across higher education. As we develop new features and functionality for the Cloud Scorecard, we look forward to ongoing collaboration with the Cornell development team to explore opportunities for contributing back to the project.
The new Finder is built on an open source Drupal platform developed by Cornell. Leveraging this open source platform aligns with Internet2's core ethos: investing in community-driven projects that deliver shared benefits across higher education.
Grush: How does Internet2's community-driven approach ensure that the Cloud Scorecard remains relevant to the evolving needs of research and higher education institutions?
Malm: I think that the success of the Cloud Scorecard to date is largely due to the fact that it is community driven. The information that we have is driven essentially by the community. I think going forward, the Cloud Scorecard is going to continue to be relevant because we, the community, have a vested stake and interest in this, and we're going to continue to devote our time, and our resources, and our energy towards making it better and ensuring that it stays relevant — because we're the ones who are going to be using it.
I think going forward, the Cloud Scorecard is going to continue to be relevant because we, the community, have a vested stake and interest in this… and we're the ones who are going to be using it.
I'm both a contributor, I would say, and a consumer. I'm contributing to the overall process of development — and then I'm also consuming the output. I've certainly got a stake in the Cloud Scorecard's success, as do so many others in the community. I think that's the key to keeping it relevant.
Grush: Finally I'd like to ask Sean, what are your wishes for the Cloud Scorecard service going forward? What have you resolved to work towards?
O'Brien: I'll share my three New Year's resolutions for the Cloud Scorecard.
Resolution number one is simply getting feedback that this resource is providing value to both the supply and demand sides. I'm encouraged that the Cloud Scorecard presents an opportunity for vendors to showcase their products and the ways their products meet higher education standards. And it's exciting that higher education institutions are able to find solutions they want as they derive value from the responses we've collected.
Resolution number two, we would love to get more vendors into the Cloud Scorecard. The more vendors who are in there, the more value I think the scorecard has in terms of people being able to see options and find services. And that's particularly helpful for early-stage vendors, if you think in terms of having a scorecard done before someone in procurement or a leader in central IT gets asked about your service. Already having your scorecard available is very advantageous.
My third resolution — and this is already in development today — is to launch natural language search for the Cloud Scorecard, to enable people to ask detailed questions about specific services or a specific technology and get a response just like they would expect from an AI chatbot. We look forward to highly conversational search and chatbot-style functionality coming soon. In fact, we're close to being ready to go live on that. Bottom line, I think that as we focus on helping users interact with the platform and keep our sights set on community-driven development, we'll get there sooner than we might expect.
[Editor's note: Image by AI. Microsoft Image Creator by Designer]