Office of Evaluations and Exams - University of Michigan

Q: Who decided to move the TQ course evaluation system online?
A: In the fall of 2006 Provost Teresa Sullivan appointed a task force to examine the feasibility of replacing the University’s existing system of course evaluations with a Web-based approach. The task force, whose nine members included faculty, administrators, and students, sought the opinions of other members of the UM community and reviewed data on online evaluations (PDF) from experimental and field studies. In March 2007 the task force issued its report on online evaluations (PDF).

The report recommended that the University develop a paperless, online course evaluation system that reproduced as many features as possible of E&E’s Teaching Questionnaire (TQ) system. In partnership with staff from Michigan Administrative Information Systems (MAIS) and CTools, E&E implemented this online system for fall of 2008.

Q: When did the TQ course evaluation system go online?
A: The system went online campus-wide in September 2008. The TQs collected during the spring/summer term of 2008 were the final ones done on paper for classes offered on the Ann Arbor campus. Teachers on the Flint campus are continuing to use paper TQ's for their teaching evaluations.

Q: What courses were affected?
A: The conversion of TQ to an online system in fall of 2008 affected about 8000 classes — all the classes in which TQ evaluations are collected on the Ann Arbor campus.

Q: How do students fill out TQ evaluations online?
A: Students will receive emails directing them to websites where they can fill out online TQ forms for the classes in which they are enrolled. The online TQ forms are virtually identical in content to printed forms. Like printed forms, the online ones ask students for both teaching ratings and comments.

Q: How do faculty obtain customized Teaching Questionnaires (TQs) using the online system?
A: Faculty and departments submit their orders online. In some departments administrative assistants submit the orders. In others faculty members and GSIs order their own TQs. Specific information about ordering is sent directly to faculty and departments early in the term.

Q: How are instructions on the use of the new system be communicated?
A: Specific information about the new system, including instructions on how to carry out the new processes, are communicated via email to faculty, students, and staff. In addition, MAIS has provided training sessions for staff members who handle TQ ordering for their departments.

Q: How do faculty and department chairs receive the results?
A: Instructors receive the same reports they did with the paper system, but in electronic rather than printed form. Reports include a summary of the quantitative rating data and images of rating sheets with student comments. As they do now, deans and department chairs, in consultation with faculty, determine who else receives rating results and comments.

Q: Why was the evaluation process being converted to an online format?
A: Online systems provide timelier feedback for faculty, better anonymity protection for students, stronger safeguards against tampering, and more ease in questionnaire design. In addition, online systems are less costly to operate. Colleges that collect evaluations online do not have to print, distribute, and collect hundreds of thousands of paper questionnaires each year.

Q: Are other universities evaluating courses online?
A: According to a Brigham Young University website, online evaluations are collected campus-wide at 16 universities, including Brigham Young, Carnegie Mellon, Northwestern, Virginia, and Yale. A Brigham Young website lists 43 other schools that collect online evaluations in one or more departments but not campus-wide, and it also lists an additional 25 schools that collect online evaluations in one or more courses but not department- or campus-wide.

Q: Have these schools reported benefits from collecting student ratings online?
A: Schools typically report lower costs with online data collection. Brigham Young University, for example, found a 50 per cent cost reduction in ratings when it moved its system online. In addition, schools often report that students make more comments online. For example, three times as many students made written comments after the Northwestern University rating system went online.

Q: What are the risks associated with the change?
A: Field evaluations at major universities have identified three major risks: lower response rates, slightly lower ratings, and easier non-university access to sensitive data.

Q: How are you addressing a possible drop in student participation?
A: Student participation is key to a successful faculty evaluation system. A communication campaign targeting students emphasizes the importance of their role in rewarding and improving teaching. Before and during the evaluation period, students and faculty will receive information encouraging participation and providing how-to guidance.

Q: How are you addressing the possible drop in the favorability of ratings?
A: Since all evaluations are in the online format, any change in rating favorability is experienced across the board. The University is monitoring performance of the new system and will make changes as needed.

Q:How are students assured of confidentiality?
A: Student anonymity is completely protected. Student identifiers are required in order to ensure that only registered students rate a class and that each student provides only one rating per class. But student identifiers are stripped from ratings and comments as soon as possible during data collection. Lists of respondents are destroyed at the end of each evaluation period, and these lists are never available to deans, department chairs, faculty or staff members. In addition, students keyboard their comments, so they do not have to worry that teachers will guess their identities from handwriting clues.

Q: What will you do about external access to the data?
A: The availability of information under the Freedom of Information Act (FOIA) will not change, and requests for information will be handled as they are currently. However, it is relatively inexpensive to copy centrally stored electronic data, and for this reason, student comments will not be stored in a central archive. Comments are sent to instructors as soon as they are compiled, and then the comments are purged from University databases.

Q: How are you guarding against tampering with the results?
A: Paper systems require many people to handle paper evaluation forms (teachers, students who collect forms and deliver them to department offices, secretaries, campus mail personnel, evaluation center personnel), sometimes with little monitoring. Online systems offer fewer risks for tampering and loss of data.




	Online evaluations FAQ Who decided to move the TQ course evaluation system online? When did the TQ course evaluation system go online? What courses were affected? How do students fill out TQ evaluations online? How do faculty obtain customized Teaching Questionnaires (TQs) using the online system? How are instructions on the use of the new system be communicated? How do faculty and department chairs receive the results? Why was the evaluation process being converted to an online format? Are other universities evaluating courses online? Have these schools reported benefits from collecting student ratings online? What are the risks associated with the change? How will you address a possible drop in student participation? How will you address the possible drop in the favorability of ratings? How are students assured of confidentiality? What will you do about external access to the data? How are you guarding against tampering with the results? Q: Who decided to move the TQ course evaluation system online? A: In the fall of 2006 Provost Teresa Sullivan appointed a task force to examine the feasibility of replacing the University’s existing system of course evaluations with a Web-based approach. The task force, whose nine members included faculty, administrators, and students, sought the opinions of other members of the UM community and reviewed data on online evaluations (PDF) from experimental and field studies. In March 2007 the task force issued its report on online evaluations (PDF). The report recommended that the University develop a paperless, online course evaluation system that reproduced as many features as possible of E&E’s Teaching Questionnaire (TQ) system. In partnership with staff from Michigan Administrative Information Systems (MAIS) and CTools, E&E implemented this online system for fall of 2008. Q: When did the TQ course evaluation system go online? A: The system went online campus-wide in September 2008. The TQs collected during the spring/summer term of 2008 were the final ones done on paper for classes offered on the Ann Arbor campus. Teachers on the Flint campus are continuing to use paper TQ's for their teaching evaluations. Q: What courses were affected? A: The conversion of TQ to an online system in fall of 2008 affected about 8000 classes — all the classes in which TQ evaluations are collected on the Ann Arbor campus. Q: How do students fill out TQ evaluations online? A: Students will receive emails directing them to websites where they can fill out online TQ forms for the classes in which they are enrolled. The online TQ forms are virtually identical in content to printed forms. Like printed forms, the online ones ask students for both teaching ratings and comments. Q: How do faculty obtain customized Teaching Questionnaires (TQs) using the online system? A: Faculty and departments submit their orders online. In some departments administrative assistants submit the orders. In others faculty members and GSIs order their own TQs. Specific information about ordering is sent directly to faculty and departments early in the term. Q: How are instructions on the use of the new system be communicated? A: Specific information about the new system, including instructions on how to carry out the new processes, are communicated via email to faculty, students, and staff. In addition, MAIS has provided training sessions for staff members who handle TQ ordering for their departments. Q: How do faculty and department chairs receive the results? A: Instructors receive the same reports they did with the paper system, but in electronic rather than printed form. Reports include a summary of the quantitative rating data and images of rating sheets with student comments. As they do now, deans and department chairs, in consultation with faculty, determine who else receives rating results and comments. Q: Why was the evaluation process being converted to an online format? A: Online systems provide timelier feedback for faculty, better anonymity protection for students, stronger safeguards against tampering, and more ease in questionnaire design. In addition, online systems are less costly to operate. Colleges that collect evaluations online do not have to print, distribute, and collect hundreds of thousands of paper questionnaires each year. Q: Are other universities evaluating courses online? A: According to a Brigham Young University website, online evaluations are collected campus-wide at 16 universities, including Brigham Young, Carnegie Mellon, Northwestern, Virginia, and Yale. A Brigham Young website lists 43 other schools that collect online evaluations in one or more departments but not campus-wide, and it also lists an additional 25 schools that collect online evaluations in one or more courses but not department- or campus-wide. Q: Have these schools reported benefits from collecting student ratings online? A: Schools typically report lower costs with online data collection. Brigham Young University, for example, found a 50 per cent cost reduction in ratings when it moved its system online. In addition, schools often report that students make more comments online. For example, three times as many students made written comments after the Northwestern University rating system went online. Q: What are the risks associated with the change? A: Field evaluations at major universities have identified three major risks: lower response rates, slightly lower ratings, and easier non-university access to sensitive data. Q: How are you addressing a possible drop in student participation? A: Student participation is key to a successful faculty evaluation system. A communication campaign targeting students emphasizes the importance of their role in rewarding and improving teaching. Before and during the evaluation period, students and faculty will receive information encouraging participation and providing how-to guidance. Q: How are you addressing the possible drop in the favorability of ratings? A: Since all evaluations are in the online format, any change in rating favorability is experienced across the board. The University is monitoring performance of the new system and will make changes as needed. Q:How are students assured of confidentiality? A: Student anonymity is completely protected. Student identifiers are required in order to ensure that only registered students rate a class and that each student provides only one rating per class. But student identifiers are stripped from ratings and comments as soon as possible during data collection. Lists of respondents are destroyed at the end of each evaluation period, and these lists are never available to deans, department chairs, faculty or staff members. In addition, students keyboard their comments, so they do not have to worry that teachers will guess their identities from handwriting clues. Q: What will you do about external access to the data? A: The availability of information under the Freedom of Information Act (FOIA) will not change, and requests for information will be handled as they are currently. However, it is relatively inexpensive to copy centrally stored electronic data, and for this reason, student comments will not be stored in a central archive. Comments are sent to instructors as soon as they are compiled, and then the comments are purged from University databases. Q: How are you guarding against tampering with the results? A: Paper systems require many people to handle paper evaluation forms (teachers, students who collect forms and deliver them to department offices, secretaries, campus mail personnel, evaluation center personnel), sometimes with little monitoring. Online systems offer fewer risks for tampering and loss of data.


		Home \| Teaching Evaluations \| Survey Services \| Placement Exams \| Classroom Exams \| Charges About E&E \| Contact Us U of M Gateway \| ©2005-09 Regents of the University of Michigan