Taming the Mega-Lecture: Wireless Quizzing
At Purdue University, faculty tested the usability of wireless PDAs for
quizzing functions in a large-enrollment course. Linked to WebCT’s testing
engine, the tiny PDAs offer giant savings in paperwork and a big appeal to students.
Each semester more than 600 students enroll in the Applied Leadership course
provided by the Department of Organizational Leadership and Supervision at Purdue’s
School of Technology. At most schools the course is generally called Introduction
to Business. In Fall 2003 the course was taught using three lecturers with four
graduate students facilitating 21 recitation sections. The enrollees represented
a variety of majors and semester classifications across campus.
Because of the time required to handle the paperwork of such a large enrollment
the professor in charge suggested it might be useful if wireless technology
could be used to save the costs of duplicating exams and quizzes and maximize
Purdue’s existing investment in the WebCT course management platform.
In the Spring 2003 semester, Purdue had been in the final stages of installing
wireless access in major classroom buildings across campus. At the same time,
the school had 35 HP Jornada 560 Series pocket PC available in its inventory.
Given this opportunity a feasibility study was done to determine if wireless
PDAs could be used to conduct WebCT-based quizzes in the classroom.
When Purdue’s information technology services group was initially consulted
with the request, the immediate answer was that it is not possible because of
WebCT’s inability to conform to the unique characteristics of the PDA.
Fortunately the IT staff persisted and ultimately arrived at a software solution
to enable the PDA to access the WebCT testing engine.
In order to begin the feasibility study many obstacles had to be overcome, including
the relocation of the equipment across campus, the identification and approval
for use of a secured storage facility, and the pre-loading of software on the
PDAs. The Applied Leadership professor then chose two recitation sections of
the class and conducted three weeks of quizzing using the wirelessly configured
Pocket PC.
Each student was given the option to select out of the study, but no one opted
out. Many students who were not part of the selected wireless sections voiced
their desire to participate.
The primary focus of the study was to determine if the wireless units could
deliver WebCT-based quizzes in an actual Purdue classroom, and if so, would
it effect the performance of students taking quizzes. The basic hypothesis was
that that no significant difference would be found between the quiz scores of
traditional classrooms and those using the wireless equipment.
The Survey
To gain more insight into the concerns of students participating in this study,
an opinion survey was developed using standard survey construction techniques,
including a control group outside the course. Students were asked to respond
to a series of statements with a 1-to-5 rating, with 1 representing strong disagreement
and 5 representing strong agreement.Below is a summary of the findings:
Statement 1: Overall I have enjoyed using the wireless PDA device to
take quizzes in class.
The response was generally positive, with a 3.66 response rate. As the control
group did not use the PDAs in any way this question did not apply to them.
Statement 2: I want to use wireless PDAs in more of my classes to take
quizzes.
There was a positive response of 3.52, with the control group registering a
3.51—a negligible difference. This seemed to indicate no negative associations
for participants using the PDAs.
Statement 3: I want to use wireless PDAs in all of my classes to take
quizzes.
The participants responded slightly more negatively than the control group and
overall disagreed with this option (participants, 2.89; control group, 2.95).
Statement 4: I am willing to purchase on my own a wireless PDA to use
in class.
The students involved in the study indicated a stronger negative response than
the control group (participants 2.28; control group with a 2.79).
Statement 5: I want Purdue to provide me a wireless PDA to use once
I come to campus.
Naturally enough, when asked if they wanted the university to provide them with
a wireless PDA both the experienced students and the control group responded
affirmatively: 4.05 versus 3.97.
Statement 6: I am willing to increase my tuition and fees up to $500
in order to receive a wireless PDA from Purdue upon enrollment.
The response was generally negative with the experienced participants indicating
a much stronger negative response (participants, 2.52; control group, 2.87).
Statement 7: Overall the wireless PDA is difficult to use.
Participants indicated that the PDA was not difficult to use, but the average
response is close to neutral (2.77), possibly indicating that for some it was
difficult while for others it was not. For this question the control response
(2.31) is particularly interesting because for them it is a hypothetical statement.
Statement 8: The screen size of the PDA is too small for use as a quizzing
tool.
Experienced participants indicated an overall affirmative (3.20). The experienced
students gave a much stronger response than was found in the control group (2.87).
Statement 9: I use a cellular phone on a regular basis.
This question indicates that both the control group and the experienced participants
are cell phone users (4.16 versus 4.36). This question was asked to gauge the
technical proficiency of the participants and to determine if they had experience
using small screens.
Statement 10: I own wireless computing equipment.
This question indicates that wireless computing equipment is not generally owned
by typical Purdue students (participants: 2.89; control group: 2.51).
Statement 11: I use wireless computing equipment at home or where I
currently live.
This question indicates that a majority of students do not yet have access to
wireless computing in their homes or dorms (participants: 2.80; control group:
2.64).
Statement 12: I consider myself a proficient computer user.
The response indicated a strong correlation between the control group and participants
on their assessment of their computer skills. Both groups gave an identical
positive response when asked if they consider themselves proficient computer
users (4.18).
Outcomes
Generally the wireless PDA units worked in accessing the Purdue WebCT testing
engine. About 10 percent of the units consistently failed to connect every time
the feasibility study was attempted. The exact nature of the failure was not
identifiable but it was most often resolved by giving the student a new PDA.
The study determined that wireless PDAs can be used to administer WebCT-based
quizzes but that possible complications must be addressed. It also found that
typical college students are agreeable to using wireless technology—but
they are not altogether willing to be forced into purchasing the equipment.
