Assessment

Hurdling Toward Campuswide E-Portfolios

Some ambitious institutions are seeking to implement e-portfolios across all departments and disciplines, yet there are many barriers to overcome before such a practice gets the full participation of faculty, staff, and students.

• By Dian Schaffhauser
• 09/01/10

When Gail Ring sat down with students in her first year on the faculty at Clemson University, she'd ask them, "What do you think of the portfolio program?" As the new director of the e-portfolio initiative for the South Carolina school, she had a vested interest in their responses. Almost unanimously they'd reply, "How do you seriously expect us to do anything with this portfolio if our professors don't mention it?"

Clearly, course integration needed to be improved. Ring also heard that advisers were clueless about portfolios. "In some cases advisers were saying, 'Don't even talk to me about it,'" Ring recalls. Although Clemson had introduced the e-portfolio as an undergraduate requirement, it had quickly become something only the students--not the faculty or staff--were apparently supposed to worry about.

Ring's experience at Clemson illustrates that the old adage, "easier said than done," applies in spades to an institutional mandate on e-portfolios. Yet, despite resistance from faculty, staff, and students, and the sheer magnitude of the effort from a technological and management point of view, some campuses are shifting portfolios away from department-specific initiatives to become institution-wide programs.

The move seems to be driven--at heart--by institutions wanting to focus more deeply and systemically on providing evidence of student mastery of learning outcomes for every area of study, something that all regional higher education accreditation agencies, in fact, expect a school to do as a part of the accreditation process.

Yet there is no requirement that a college or university use portfolios (electronic or otherwise) to provide such evidence. The accrediting agencies allow various types of assessments to demonstrate mastery, including capstone tests and assignments, licensure examinations, and student performances, as well as portfolios. Not surprisingly, many higher ed institutions do use portfolios for selected areas of study, and e-portfolios, in particular, are emerging as an optimal way to manage these collections of student work over time.

But institutions like Clemson and Virginia Tech believe that integrating e-portfolios into a campuswide assessment system is, as Ring puts it, "the most effective way to assess our students' understanding of [key] competencies." So these schools are taking on the laborious challenge of implementing e-portfolios across departments, colleges, and disciplines. They believe these efforts are creating a student-centered learning environment that will transform the educational experience at their universities.

Competency-based Assessment
Clemson introduced e-portfolios as an undergraduate requirement in fall 2006 after a faculty revision of the university's general education competencies. That rework produced seven main competencies, such as cross-cultural awareness and ethical judgment, which were themselves composed of 22 sub-competencies.

At the same time, the provost appointed a task force of administrators, faculty, staff, and students to figure out how those competencies could be assessed. Their verdict: By the time a student graduated from Clemson, he or she had to complete a portfolio with a piece of evidence for mastery of each of those 22 competencies.

After the portfolio requirement was set, however, "not a lot of attention was given to integration, supporting students with a place to go for help, online tutorials, or things like that," says Ring. By the time she arrived a year later to run the program, two elements were in place: Every faculty member teaching a general ed course had identified the competencies in their syllabus; and Blackboard, the university's learning management system, had been designated as the mechanism for maintaining the portfolio. But that's where progress had pretty much stopped.

Ring began introducing herself to faculty teaching freshman courses--particularly those who were in the bigger colleges on campus. She gave e-portfolio integration workshops to faculty in specific course areas--starting with engineering, moving to business, then nursing--showing what kinds of assignments might make up a student portfolio: research papers, spreadsheet analyses, design projects, PowerPoint presentations, and so forth.

But those efforts only had partial success. "Again, in many cases progress stopped," Ring sighs. "It wasn't picked up again until students were approaching graduation, when it suddenly became more visible."

The program lacked some essential elements, Ring concluded, including a way for students to give and receive systemized feedback. "Looking at your work with your faculty mentor or adviser and even peers is essential," Ring explains. "That's what makes a portfolio a portfolio."

So Ring focused first on developing a peer-support program that provides students with the framework to offer each other productive feedback. Ring recruits undergraduate e-portfolio enthusiasts and assigns them to a "creative inquiry group," corresponding to the main competency areas. After rigorous training, they evaluate examples of student work--called artifacts--under the watchful eye of the faculty member in charge of that area. Each artifact is scored on a scale from one to four, four being a "fantastic demonstration of competency," and one being, "This doesn't demonstrate the competency."

Ownership of the Technology
Even as she was guiding students on how to be good peer reviewers, Ring was stymied by another hurdle, this one related to technology. During her first year on campus, students overwhelmingly told her she needed to improve the Blackboard tool being used to maintain the e-portfolios.

"They really felt like the tool we were using didn't give them ownership," she explains. "They wanted to keep their materials wherever they chose." But the university wanted private assessment of what was in the portfolio, which meant it needed some level of access and control over student materials.

Then something fortuitous (in hindsight) happened. In summer 2008, during a week in which faculty were to look at the contents of student e-portfolios, the entire system collapsed. "We had no access to student artifacts," Ring recalls. Suddenly, the university had an excuse to ask, do we rebuild the system that nobody likes, or do we go in another direction?

Simultaneously, Clemson was planning a fall 2008 move to Google Apps for Education, which gave Ring the opening she needed: The university moved off of the e-portfolio tool in Blackboard and students were empowered to use anything they wanted to maintain their artifacts (though most use Google Sites, Google's free Web site creator).

To address access and control concerns, a computer science class project at Clemson created a utility called CUePort to provide an interface between a student's Google Site, where the artifacts exist, and the general education competencies, which reside within CUePort. The student links the artifacts to the competencies, adds rationale statements, and, where necessary, groups artifacts around competencies. The same utility provides feedback functionality and administrative access for pulling program data that can be used for institutional accreditation.

But even as progress was being made, it became clear that the program was collapsing under the weight of 22 competencies. Ring urged the university to reassess the number and ultimately a special task force lowered the total number of general education competencies to a manageable eight, which will be introduced this fall.

What's still missing, however, is faculty feedback to the students. "More and more we're seeing professors wanting to be engaged in the process," says Ring. "They'll look at artifacts and can leave comments on Google Apps sticky notes. But we haven't built into the system a way for faculty to give feedback."

After her third year on the job, Ring warns that "we're still not there yet." But she points out that many of the faculty are revising both their syllabi and their courses in light of the portfolio requirement, and that students are identifying better artifacts and writing better rationale statements.

She notes, "We trusted our students to self-select--to select the evidence they believe best fits the competency. They didn't always make the right choice. But through those choices, those decisions, and then through the justifications in those rationale statements and the feedback they got from peer reviewers, they got better."

She concludes: "It's been incredibly challenging to try to make this happen across the entire university, but the support and student centeredness is there."

Getting Buy-In From Everyone
"We don't reflect." That was the response from a Virginia Tech engineering student when presented with an explanation about the school's new ePortfolio Initiative, a program devised to formally facilitate the use of e-portfolios university-wide.

The initiative was launched in 2008 as part of an institutional plan for educational improvement, in part triggered by the school's upcoming 2010 accreditation review by the Southern Association of Colleges and Schools.

While the Blacksburg, VA, institution had been doing portfolio pilots since 2005, there had been "no mandate that programs at Virginia Tech even had to have learning outcomes," says Marc Zaldivar, the initiative's director. So when Zaldivar and his team began pitching portfolios as a useful technology for documenting the achievement of learning outcomes, most faculty and students had some prior understanding of the tool.

But, as the engineering student's attitude illustrates, the colleges and departments at Virginia Tech have varied dramatically in their acceptance of the portfolio initiative.

Time, Training, and Understanding
The effort faces three challenges, says Zaldivar. The first is time: Students are already really busy in their classes and instructors don't have spare moments to fit in something new. Second, there's a learning curve involved in picking up new technologies--in this case the e-portfolio tools in Sakai, which this fall will also become Virginia Tech's learning management platform. Third, faculty members don't always understand how to turn their lessons into portfolio activities.

The team has addressed these challenges head-on in multiple ways. For example, two major portfolio proponents on campus--the English and dietetics departments--have created peer mentoring teams: groups of students who talk to other students, speak at faculty workshops, and lay out the benefits of the e-portfolio program.

Every semester and summer for the past few years, the Faculty Development Institute, which offers training on technology and teaching topics, has given Zaldivar a forum for explaining the e-portfolio program. Since every faculty member must attend once every three years in order to get a new computer, "We almost have a captive audience," he says. "Over a three-year cycle, by giving workshops in these training tracks, we have exposure to almost every member of the faculty on campus."

The tool used for the work--Sakai's Open Source Portfolio--has aided the effort by providing a good deal of flexibility. Its matrix format shows learning outcomes at the side and a timescale at the top, which can act as a workflow for students in a course, allowing them to submit course assignments, reflect on those assignments, and get feedback and evaluations. Or the matrix can act as a simple checklist grid to show the completed steps in a program.

Faculty are encouraged to join the portfolio fray by starting with the highest educational priority in a given course, for which the portfolio team will build an appropriate template. Zaldivar says the robustness of the toolset has given his department a means to address a huge diversity of e-portfolio needs on campus. Those instructors who want to focus simply on assessment rubrics have that capability. Others who want to expand and include student reflection or provide students with a way to put forth a professional face with their artifacts can also be supported.

Another practice is to encourage students to keep multiple portfolios. The content is actually all maintained in the same repository, but each course may have its own view of that collection. "A lot of faculty are overwhelmed by the [prospect] of having to dig through a large portfolio to find the work that is needed. So if you need one portfolio that's small for one teacher, that's very easy to do," Zaldivar says.

An added benefit of that multiple-portfolio functionality, adds Teggin Summers, assistant director of the program, is that students also can share the same artifacts or other content among portfolio views.

Starting in fall 2010, the institution will introduce 1,200 first-year students (roughly one quarter of the incoming class) to the e-portfolio experience. Over the course of the year, all of those students will have three outcomes to demonstrate mastery of across subject areas: problem solving, inquiry, and integration. For problem solving, for example, students will need to share artifacts and reflections focused on defining a problem, identifying problem-solving strategies, and proposing solutions or hypotheses. Integration calls on students to provide evidence that they can connect relevant experience and academic knowledge, make connections across disciplines, and reflect upon and assess themselves as learners. Each participating department or school will address these outcomes with different assignments; then the students will write reflections about how they improved in each outcome over their first year. During the next five years, the number of participants will scale up until the e-portfolio program affects all 5,000 first-year students.

Will these efforts convert those seemingly intractable engineering students? Not in the first go-around. "A lot of them want simple numbers to do assessments," concedes Summers. "They're not comfortable with the qualitative pieces of this. They don't understand the validity or the process. They're resistant."

But that doesn't mean that the ePortfolio Initiative team is daunted. Virginia Tech has invested in the idea that e-portfolios promote transformative education, lifelong learning, student-centered instruction, and a culture of assessment, says Zaldivar. "We know there are many challenges, but this is the growing vision at VA Tech. We'll be getting there eventually."