Teaching and Learning Evaluation Results Summary
To perform the actual evaluation of teaching and learning, each faculty and instructional support member of our evaluation team was given access to both a standard Moodle 1.9 and Blackboard Learn 9.1 environment, including blank and populated courses, and access to each other’s courses as faculty and students. The Learn environment included the Content module, although many of its collaborative and content-sharing features were not formally evaluated. No formal training was given in each LMS, but online tutorial references were supplied to evaluators including step-by-step tutorials and quick videos.
Team members used our LMS Evaluation Rubric (PDF) to evaluate each LMS feature or capability on functionality as well as easy of use. All evaluators were asked to select those features they use, or would like to use, because the list was too exhaustive to expect active faculty to try everything, including LMS features with which they were not familiar. Comments on each feature and overall for each LMS were also solicited. The six-week timeline for this evaluation process (detailed in a previous post) was extended several weeks due to our busy semester, and in the end not all team members elected to submit completed rubrics. However, we feel the end results do represent the overall tenor of the evaluation team.
These results were presented to the evaluators in a meeting in December, 2010, with the understanding that this data would help to shape an LMS recommendation but it did not necessarily dictate what that recommendation would be. The final recommendation would take into account this data as well as factors such as cost, staff impact, technical requirements, license structure and governance.
The Evaluation Verdict
In short, Blackboard Learn 9.1 had a slight edge in both the Ease of Use and the Functionality measures over Moodle, in almost all feature areas.
Several faculty evaluators expressed a preference for either Moodle or Blackboard, but nearly all members of the group felt that both systems could capably serve the teaching and learning functions of Chico State.
Now, let’s have some charts!
First, here’s a graph showing our evaluators’ ratings of the top 10 tools used by faculty on our campus (according to the faculty survey):
Next, a graph showing our evaluation team ratings of the top ten tools used by students (from the student survey):
And here are the results across all teaching and learning functions evaluated. The order of features is taken from the evaluation rubric’s groupings.
Here are the functionality and ease-of-use results separated, and displayed in a way which shows the preference via a center point on each two-color bar.