Taming the Mega-Lecture: Wireless Quizzing

At Purdue University, faculty tested the usability of wireless PDAs for quizzing functions in a large-enrollment course. Linked to WebCT’s testing engine, the tiny PDAs offer giant savings in paperwork and a big appeal to students.

Each semester more than 600 students enroll in the Applied Leadership course provided by the Department of Organizational Leadership and Supervision at Purdue’s School of Technology. At most schools the course is generally called Introduction to Business. In Fall 2003 the course was taught using three lecturers with four graduate students facilitating 21 recitation sections. The enrollees represented a variety of majors and semester classifications across campus.

Because of the time required to handle the paperwork of such a large enrollment the professor in charge suggested it might be useful if wireless technology could be used to save the costs of duplicating exams and quizzes and maximize Purdue’s existing investment in the WebCT course management platform.

In the Spring 2003 semester, Purdue had been in the final stages of installing wireless access in major classroom buildings across campus. At the same time, the school had 35 HP Jornada 560 Series pocket PC available in its inventory. Given this opportunity a feasibility study was done to determine if wireless PDAs could be used to conduct WebCT-based quizzes in the classroom.

When Purdue’s information technology services group was initially consulted with the request, the immediate answer was that it is not possible because of WebCT’s inability to conform to the unique characteristics of the PDA. Fortunately the IT staff persisted and ultimately arrived at a software solution to enable the PDA to access the WebCT testing engine.

In order to begin the feasibility study many obstacles had to be overcome, including the relocation of the equipment across campus, the identification and approval for use of a secured storage facility, and the pre-loading of software on the PDAs. The Applied Leadership professor then chose two recitation sections of the class and conducted three weeks of quizzing using the wirelessly configured Pocket PC.

Each student was given the option to select out of the study, but no one opted out. Many students who were not part of the selected wireless sections voiced their desire to participate.

The primary focus of the study was to determine if the wireless units could deliver WebCT-based quizzes in an actual Purdue classroom, and if so, would it effect the performance of students taking quizzes. The basic hypothesis was that that no significant difference would be found between the quiz scores of traditional classrooms and those using the wireless equipment.

The Survey
To gain more insight into the concerns of students participating in this study, an opinion survey was developed using standard survey construction techniques, including a control group outside the course. Students were asked to respond to a series of statements with a 1-to-5 rating, with 1 representing strong disagreement and 5 representing strong agreement.Below is a summary of the findings:

Statement 1: Overall I have enjoyed using the wireless PDA device to take quizzes in class.
The response was generally positive, with a 3.66 response rate. As the control group did not use the PDAs in any way this question did not apply to them.

Statement 2: I want to use wireless PDAs in more of my classes to take quizzes.
There was a positive response of 3.52, with the control group registering a 3.51—a negligible difference. This seemed to indicate no negative associations for participants using the PDAs.

Statement 3: I want to use wireless PDAs in all of my classes to take quizzes.
The participants responded slightly more negatively than the control group and overall disagreed with this option (participants, 2.89; control group, 2.95).

Statement 4: I am willing to purchase on my own a wireless PDA to use in class.
The students involved in the study indicated a stronger negative response than the control group (participants 2.28; control group with a 2.79).

Statement 5: I want Purdue to provide me a wireless PDA to use once I come to campus.
Naturally enough, when asked if they wanted the university to provide them with a wireless PDA both the experienced students and the control group responded affirmatively: 4.05 versus 3.97.

Statement 6: I am willing to increase my tuition and fees up to $500 in order to receive a wireless PDA from Purdue upon enrollment.
The response was generally negative with the experienced participants indicating a much stronger negative response (participants, 2.52; control group, 2.87).

Statement 7: Overall the wireless PDA is difficult to use.
Participants indicated that the PDA was not difficult to use, but the average response is close to neutral (2.77), possibly indicating that for some it was difficult while for others it was not. For this question the control response (2.31) is particularly interesting because for them it is a hypothetical statement.

Statement 8: The screen size of the PDA is too small for use as a quizzing tool.
Experienced participants indicated an overall affirmative (3.20). The experienced students gave a much stronger response than was found in the control group (2.87).

Statement 9: I use a cellular phone on a regular basis.
This question indicates that both the control group and the experienced participants are cell phone users (4.16 versus 4.36). This question was asked to gauge the technical proficiency of the participants and to determine if they had experience using small screens.

Statement 10: I own wireless computing equipment.
This question indicates that wireless computing equipment is not generally owned by typical Purdue students (participants: 2.89; control group: 2.51).

Statement 11: I use wireless computing equipment at home or where I currently live.
This question indicates that a majority of students do not yet have access to wireless computing in their homes or dorms (participants: 2.80; control group: 2.64).

Statement 12: I consider myself a proficient computer user.
The response indicated a strong correlation between the control group and participants on their assessment of their computer skills. Both groups gave an identical positive response when asked if they consider themselves proficient computer users (4.18).

Outcomes
Generally the wireless PDA units worked in accessing the Purdue WebCT testing engine. About 10 percent of the units consistently failed to connect every time the feasibility study was attempted. The exact nature of the failure was not identifiable but it was most often resolved by giving the student a new PDA.

The study determined that wireless PDAs can be used to administer WebCT-based quizzes but that possible complications must be addressed. It also found that typical college students are agreeable to using wireless technology—but they are not altogether willing to be forced into purchasing the equipment.

Featured