Blog lag

My blogging sure has taken a hit as of late. Things have been busy at work (migration from Blackboard 5.5 to moodle 1.5 — and all of the customizations to moodle, course evaluations online — both a new set of questions and doing it online) and at home (our 2nd story is about to be removed and rebuilt anew).

The online course evaluations have been very interesting. It’s a real intersection of students, faculty and technology. You have the factors of student attitudes to course evaluations — are they anonymous? do the faculty care? does my opinion matter? And then the faculty attitudes towards evaluations — what if only students with negative opinions do them? non-tenured faculty worry about tenure decisions. And then the technology factor of being online adds the new variable of response rate. Doing paper evals during class gives a captive audience — the evals are optional, the faculty member has to leave the room, but the time allotted varies. The handwriting issue also comes into play for anonymity. Doing them online makes it easier to not do them. There hasn’t been extensive research on doing course evaluations online, but there are some articles I’ve found.

First of all, some effective practices are emerging. The TLT Group’s Flashlight Program BeTA Project has some insights to successful online evaluations. What is interesting is that several of the articles I found on the subject echo similar findings. Generally, institutions awkwardly start doing online evaluations. Sometimes things go bad, they try a few things to improve response rates, and then find things that work. These practices match quite closely the BeTA findings above.

What I find interesting is that the institutional culture around evaluations seems to influence their success when taken online. I’ve learned from smarter people than I that the social aspect can overwhelm a technology project. This is why Dr. Pike used Bolman and Terrence’s 4 frames (structural, political, human resource, cultural) when approaching the course evaluation redesign last summer (see our paper for more).

Back to some resources if you’re looking to move your institution to online course evaluations. I’ve tried to link to them all and note the institutions. Some focus on response rates, some are more general. Some have bibliographies that can lead you down more paths.

• “Student Feedback on Teaching: Online! On target?” – Cummings and Ballantyne, Murdoch University.
• “Student Feedback on Teaching: Reflections and Projections” – Refereed Proceedings of Teaching Evaluation Forum 28 – 29 August 2000, Perth, Western Australia
• “Lessons Learned From Online vs. Paper-based Computer Information Students’ Evaluation System” – Liegle and McDonald, Georgia State University
• “On-line vs. Paper-and-Pencil Surveying of Students: A Case Study” – Handwerk and Carson, University of North Carolina at Greensboro (ERIC document RIEAAPR2001)
• “Administration of Web versus Paper Surveys: Mode Effects and Response Rates” – Matz, University of North Carolina (ERIC document ED439694).
• “Online Student Evaluation of Instruction: An Investigation of Non-Response Bias” – Thorpe, Drexel University
• “Findings from the 2001 Pilot Administration of Your First College Year (YFCY): National Norms” – Sax, University of California, Los Angeles
• “Online Course Evaluations: Lessons Learned” – presentation at The CALI Conference for Law School Computing 2005
• “Online Student Evaluation of Instruction” – California State University Fresno TLT
• “The Effects Of Delivery Mode Upon Survey Response Rate And Perceived Attitudes Of Texas Agri-Science Teachers” – Fraze + more, Texas Tech University
• “Online Student Course Evaluations: Review of Literature and a Pilot Study” – Anderson + more, University of Kentucky
• summary of online course evaluation trial – Finke from email list archive
• “Summer 2004: Summary of Response Rates for Online-Delivered Course Evaluations” – Conn + more, Northern Arizona University
• “Online Course Evaluations: One Institute’s Success in Transitioning from a Paper Process to a Completely Electronic Process!” – presentation at Association for Institutional Research Forum 2005
• “An Experimental Investigation Of Student Response Rates To Faculty Evaluations: The Effect Of The Online Method And Online Treatments” – Dommeyer + more, California State University
• “Incentives The Key Ingredient for Successful Web-based Course Evaluations” – Murphy, University of California