Evidence of Learning Online: Assessment Beyond The Paper
Discussions of technology strategy and planning for new media at colleges and universities are informed by many factors of higher education culture and the way its core constituents--faculty and students--work and learn. One rapidly evolving area is online assessment, whether for fully online programs or for blended learning environments. Here, learning designer Judith Boettcher examines online assessment strategies beyond the traditional end-of-term paper.
In designing courses for online environments we have been somewhat successful at scuttling closed-book exams for assessing student learning--largely due to the challenges of monitoring exams. However, we still appear to be stuck in another very entrenched mode of assessing student learning: research papers and project reports.
Why is the traditional paper so prevalent in assessment, and how can we move beyond it to alternative evidence of student learning? And how can we leverage technology and new media in our assessment strategies?
Students often dread writing papers and respond enthusiastically to alternatives that demonstrate what they know and understand. Faculty, too, often secretly--or not so secretly--cringe at the thought of reading endless numbers of papers. Faculty are searching for less burdensome assessment modalities, while striving to be effective and fair at determining what in the mountain of student papers possibly demonstrates real understanding and learning growth.
We also know this: While we may want choices beyond the classic research papers, we don’t want to move beyond writing. Writing is a core competency and we are now writing more than ever. Happily, choices of “writing places” abound and are evolving, especially with Web 2.0 tools and new media. Writing places can be collaborative, public, visible, fluid, useful, or fanciful. Writing places can be ephemeral or enduring. Writing well in blogs, wikis, social networking sites, reviews, and news commenting that we do as participants in 21st century crowd work requires thoughtful and appropriate communication skills.
So it is time, I think, to update our thinking towards “writing assignments.” What kinds of writing do learners need to do in their chosen careers, lives, and professions? Much of the writing will rely even more on critical thinking and research skills, but it will result in quite different types of products. As one step towards thinking beyond the paper, let’s consider why papers of all kinds have always been so popular as evidence of learning.
Papers are popular because writing requires skills such as:
· Processing data and information
· Critical thinking about issues and ideas
· Developing familiarity in a particular domain of knowledge
· Creativity in assembling ideas and creating something new
· Communication skills--thinking about an audience, ability to write professionally
· Analysis of a topic or of issues
· Predictive ability based on ideas and data
When designing an online or blended course, then, the question might be, “Which of these skills and those closely related skills are core for your discipline and map to your learning goals and outcomes? The companion question to be addressed is, “What type of writing product or writing space will provide the best place for learners to practice developing those skills?
Design Themes in the Background: Cognitive Surplus and Creating while Learning
A useful design theme for thinking about new writing spaces is “cognitive surplus.” This term was popularized/promulgated by Clay Shirky, a writer on the social and economic effects of Internet technologies. Shirky describes cognitive surplus as “the shared, online work we do with our spare brain cycles.” For example, cognitive surplus is what is making Wikipedia possible and is the energy behind most of the open source movement. Here’s why this is important. The motivation for doing shared work with those “spare brain cycles” is simple: We, as people, like to create and we like to share.
So we now have an opportunity. We have the tools to easily integrate learner motivation, autonomy, and relatedness into teaching and learning experiences while also retaining the focus on writing and communicating well. Building on the concept of cognitive surplus, if we can tap into learners’ motivations for creating and sharing, learners will invest more time into their learning, and learn more.
And an emerging trend in online courses is for learners to create while learning. Some of this creation work focuses on current course topics, resulting in mini-conferences and expansion of course materials. Other creative course work flows forward to future learners, to the community, and to other groups.
Alternatives to the Research Paper
So what are the alternatives to the traditional research paper? What alternatives both integrate the creating and sharing motivations associated with “cognitive surplus” and also familiarize learners with the kinds of work products they might encounter and produce in their discipline? Here are a few to consider:
Professional Work Products
· Written and audio communications of all types, such as press announcements, white papers, briefs, summaries, memos, project management documentation
· Creating and planning news events, such as announcements, interviews, or regular updates of interests, such as podcasts
· Setting up personal or group blogs within different contexts of leadership, business, etc.
· Setting up wikis for team projects, areas for monitoring developments
· Hosting a discussion thread and preparing a summary document acknowledging contributions and connecting the ideas
· Developing an annotated set of resources for a task force addressing a company or institutional challenge
· Developing a set of questions for an expert interview
· Developing a set of alternative strategies for responding to crises
Interviews
The interview medium is a very flexible communication tool and can be used by both faculty and students for demonstrating understanding and eliciting the state of concept development. Here are some possible strategies that can require research, critical thinking, and writing.
· Learners identify an expert or a person of interest to them in a particular field germane to the course and then prepare the interview questions, do the interview, and then post the results
· Learners identify and interview the author of a textbook or article closely related to the course, possibly updating information critical to the course
· Learners identify a person of interest, such as an imagined or real leader and interview that person (a related strategy is for learners adopt the persona of a person and “role play” that person)
· Turn the tables and have the students prepare to interview you, the course expert, on a certain topic to see if they can “stump” you
Of course, some of these ideas (involving interviews) would not be appropriate alternatives to the research paper, but they can be good alternatives to shorter papers or assignments.
Audio, Video, and Visual Projects
What about other media such as audio and video projects? Today’s learners live surrounded by audio and video and the tools that make it possible for everyone to create and produce audio and video products. Here are some of the possibilities with audio and video spaces.
· Podcasting resources now are very common so learners are familiar enough with the format to embrace creating audio and video podcasts of their own
· Video shorts and ad hoc documentaries engage learners and draw in their friends and families
· Creating and posting short reports via VoiceThread is another “writing space” to consider as are Flickr, YouTube, and Slideshare
Blogs
Blogs are a very underutilized writing space. Blogs share many characteristics with journals and thus can capture snapshots of what learners are thinking, and when; plus they often can also capture the sources of some of their thinking. Blogs help learners understand the growth cycle of learning new concepts and how and why they think the way they do. Here are some ideas on how blogs, both personal and class, might be used.
· Personal commentary and self-reflection
· Capturing thought processes and generating new ideas
· Assist learners in finding their “voice”
· Places for collegial commenting and suggestion
· Making thinking visible: analysis, synthesis, application
Wikis
Discipline-specific wikis are evolving as potentially awesome resources developed collaboratively with built-in checks and balances, often using our “cognitive surplus” as identified by Shirky. Course wikis can create additional sets of resources for courses, including resources on future topics that are or will be impacting a discipline. Here is just a sampling of how incorporating wikis might become a good substitute for a research paper for a small group of learners. Part of the power of the wiki is that there is room for multiple perspectives and ideas, and room for learners to exercise various project roles.
· Collaborating on group and team projects of all kinds
· Capturing and developing ideas for solving critical problems and case studies and simulations
· Developing “featured” Bronze star Wikipedia articles on specific topics in particular disciplines
Toward Paper-less Assessment
End-of-course activities where learners share their projects can be much more stimulating and energizing when learners have more choices in projects that enable them to tap into their creativity and their enjoyment in learning and sharing. Such alternatives also mean students and faculty reach out beyond the course to engage other experts and to provide service and expertise to others while bringing excitement and possibly even accountability back into our learning experiences. Campus technology planners as well as faculty should consider how evolving assessment modalities are incorporating technology and new media. We are moving toward term paper-less assessment!
[Editor’s note: Judith Boettcher will offer a half-day, pre-conference workshop on “Principles and Practices for Online Courses that Engage Learners” on July 25, 2011 at the Campus Technology 2011 conference in Boston.]