Evaluation

<< Return to Table of Contents

Purpose

The main purpose of Engage’s evaluation was to provide campus with a better understanding of methods of good practice in facilitating group work and how technology can enhance group work and alleviate the challenges of group work. In particular, the evaluation was designed to determine:

  • If there was a change in student attitudes towards group work due to the use of technology or due to research-based facilitation strategies.
  • If and how collaborative technologies alleviate challenges of group work.
  • If and how collaborative technologies enhance the group work process.
  • The affordances and drawbacks of various collaborative tools
  • Methods of good practice for designing, implementing, and assessing technology-enhanced collaborative activities

The evaluation process also provide information on the award format in an effort to determine what worked well for instructors and consultants, and what could be improved for the next Adaptation Award Program.

Methodology

The data that follow were collected through pre- and post- student and instructor surveys (Appendices 1-4), which included both open- and close-ended questions. The instructor pre-survey measured attitudes toward group work assignments prior to the Engage Award. The instructor post-survey measured attitudes toward group work assignments following the Engage Award, gathered individual project details, and provided feedback on the award format. The student pre-survey measured attitudes toward group work prior to the Engage award, determined challenges encountered in past group projects, and gauged previous experience using collaborative technologies for group work. The student post survey measured attitudes toward group work following the Engage Award, gathered specific project details, and sought evidence of how collaborative technologies and instructor facilitation methods influenced the common challenges of group work.

Instructors and Teaching Assistants who did their collaborative group project in the fall, were emailed a link to the pre-survey in early September 2008 and were emailed a link to the post survey in December 2008. Instructors who did their projects in the spring, were emailed a link to the same pre- and post-surveys in January and May 2009. Lastly, instructors who completed the project in the summer were emailed a link to the pre- and post surveys in May and August 2009. Reminders were sent to the instructors 1 and 2 weeks after the initial email invitation and the surveys were kept open for approximately 3 weeks. A total of 50 instructors were invited to participate in the surveys. Thirty-nine instructors completed the pre-survey, 42 completed the post-survey and 36 completed both. The data from both surveys were exported to SPSS for analysis. The quantitative data were analyzed in SPSS and the qualitative data were coded and analyzed by hand.

Similarly, students completed pre- and post- surveys at the start and end of the semester. One of the instructors taught a year-long course, and the students in this course took the pre-survey in the Fall and the post survey in the spring. Each instructor was sent a draft of the pre- and post-surveys and had the opportunity to add questions specific to their course to one or both survey(s). Of the 34 instructors whose students participated in the pre- and post-surveys, 12 of them chose to add questions to one or both of the survey(s) for their course. After the instructor approved the surveys, they were entered into a web survey tool, SurveyMonkey. The links to the student surveys were sent to the instructor and the instructor sent the link to the students in an email or posted it to their course website. Reminders were sent to students 1 or 2 times after the initial invitation and the survey remained open for 2-3 weeks, depending on the course. A total of 1360 students were invited to participate in the surveys. Nine hundred and forty-nine students completed the pre-survey, 802 completed the post-survey, and 660 completed both. The data from all the surveys were exported to SPSS and combined into one file for analysis. The quantitative data were analyzed in SPSS and the qualitative data were coded and analyzed by hand.

Next >>