Marshall Online Teaching Evaluation System
Suh-Pyng Ku, USC Marshall School of Business

The USC Marshall School of Business launched a new Online Teaching Evaluation System in Fall 2002, which replaces the traditional in-class, paper-based, manual-oriented teaching evaluation. The online system is flexible, anonymous, secure, efficient, customizable and personalizable and is designed to be database-driven so data can be converted to useful information instantaneously with strong data mining and reporting capabilities that can aid effective decision-making. The demand for immediate teaching evaluation feedback has long been an issue. The old paper evaluation system required surveys to be shipped off site and results took months to receive. Faculty were unable to improve their courses based on feedback received from the students without significant delay because the evaluations were not returned until well after the new semester was underway. Moreover, the time and effort it took for faculty to sift through the evaluations to improve their courses was significant with the old paper-based evaluations. And the storage required to accommodate the volume of paper evaluations was a huge burden. The new system provides feedback in a legible, typed format, with concise, on-demand reports that can be viewed online or be printed.

Unlike the traditional in-class, paper-based, written teaching evaluation, the online system allows students to provide feedback on the course and the instructor(s) online anytime during the 8 days to 2 weeks preceding their final exams, and anywhere that is convenient without peer pressure. The new instrument includes learning objectives that faculty can select as important or essential to their particular course. Students are only asked to assess their progress on these objectives. Faculty are able to customize their evaluations for varying learning objectives, something that was unattainable in the standardized paper-based teaching evaluation system Marshall has used for the past several years. In addition, there are a set of standard items for all courses that deal with students' perceptions of course management, fairness in grading, organization of the course, accessibility of the faculty member outside of the class, and how demanding the class was perceived to be. A number of these questions are open-ended and students are able to type their comments directly into the evaluation system. Finally, faculty are able to customize their evaluations by adding additional items that may be of particular relevance to their specific course. Additionally, once a faculty member for a given course enters learning objectives, they become the default objectives for the course in future semesters unless the faculty member wishes to revise them.

The online course evaluation system was created within the context of a larger online survey system, which will also allow faculty members to conduct survey research online. The new system also strives to raise student response rates, as well as provide on-demand reports to faculty and administration.

At Marshall, we consider student feedback through teaching evaluations a critical component to our continuing success in meeting our educational mission. We believe it is every student's responsibility to provide feedback on his/her learning experience in all Marshall courses. Our goal in developing this system is to leverage existing technology to improve our business process and provide faculty with timely information so they can improve their instruction and better meet learning outcomes. We also hope to give students a chance to answer evaluation questions without time pressure and therefore to provide more thoughtful feedback to their instructors.

The largest challenge we faced in developing the online evaluation system was accommodating the needs of the various Marshall constituents. Marshall offers four MBA degree programs, two Masters degree programs in Accounting, one Ph.D. program, and two undergraduate programs. Many of these programs, especially at the masters' level, are on unique class schedules that do not follow the typical semester calendar. Because of the complicated nature of program structures and scheduling, it was a very time consuming and labor-intensive task to prepare, schedule, deliver, administer, collect, and distribute the paper-based teaching evaluations and results. With input from the Marshall Registrar's Office, we developed a Marshall master course calendar coinciding with the Fall 2002 online evaluation rollout. The calendar now serves as the foundation for the Online Teaching Evaluation System and provides a database-driven system that is standardized across classes. We no longer need to worry about getting written forms from the administrative offices to the classroom and vice versa. In addition, the online system offers the added security of allowing students to fill out only one evaluation, and ensures that the individual is officially enrolled in the course. And, of course, the online evaluation system increases class contact time.

We hope the new online system will improve our response rate from the current 75% to our target of 90%. During our pilot of the online system this summer, we received a response rate of 92%. Although the system provides anonymous feedback, we are able to monitor which students have and have not completed the evaluation and can build incentives for completion. Automatic email reminders sent to students who have yet to fill out their course's survey assist in raising the response rate.

The system functions as follows. Faculty enter their course learning objectives at the beginning of the semester. Students are also sent an email introducing them to the new system at the beginning of the semester. Students are sent a second email near the end of the term (or semester), notifying them that they can access their course evaluation the following morning. The online system is available to them for a period of 8 days to 2 weeks prior to their final exam depending on the length of the course. During the evaluation period, faculty can obtain a list of students who have not completed the evaluation (but not the actual responses of students) and encourage responses. The evaluation is closed on the day before the course final exam. The evaluation results are compiled automatically on the daily basis and summary reports are made available to the faculty and administrators after the course grades are submitted.

For the protection of our data, our servers are backed up nightly and a solid disaster recovery plan is in place in the event of a technical system failure. The online system adds security features that were not in place in the paper system. There is no longer any concern of evaluations being tampered with before delivery to administrative offices and the logistical challenge of receiving all evaluations is eliminated. The new system allows only enrolled students to complete a course survey, and it prohibits multiple evaluations from a single individual. A time-out feature on the system protects students and their work in the event that they walk away from a lab computer while still logged on to their course evaluation. Additional safeguards of the system help to further protect both faculty and students, such as, faculty are able to view their survey results only after grades are released; and the evaluation for a particular class is closed prior to the start of the final exam.