Evaluation Overview

Evaluation of Awards & Projects

Engage collaborates with campus partners and faculty awardees to evaluate the effectiveness of instructional technologies. The aim of Engage evaluation is:

  1. To measure the effectiveness of instructional technologies in enhancing student learning and/or student’s perception of learning across a variety of disciplines and learning activities.
  2. To measure how the effective use of technology for teaching and learning aligns with various pedagogical frameworks and campus initiatives.
  3. To share methods of good practice for effectively implementing technology into teaching.
  4. To measure the impact of Engage Award Programs on teaching and learning (with and without technology) on our campus and beyond.

Innovation Awards

2007-2009: Simulations and Games, Round 1
Evaluation of the Simulations and Games projects includes a variety of methodologies from focus groups to experimental studies. The purpose of evaluation is to gather evidence of the student experience to inform future simulation and game design and measure the effectiveness of instructional games and simulations on student learning. View the evaluations from Round 1 here.

Engaging to Learn: Simulations & Games 2007-2012

October 2007

Simulations and Games Phase II Engage Staff PPR Report (.pdf)
Simulations and Games Phase II Awardee Survey Results (.ppt)

June 2008

Simulations and Games Phase III Engage Staff PPR Report (.pdf)
Simulations and Games Phase III Awardee Survey Results (.pdf)

2008 Play@Pyle Follow-up Survey Results (.pdf)
2008 Play@Pyle Highlights (.pdf)

Simulations and Games Round 1 Projects
Simulations and Games Round 2 Projects

2010 Round 1 Simulations and Games 1-pager(.pdf)

2010 Round 2 Mid-Project Awardee Survey Results (.pdf)

LEAP – Essential Learning Outcomes and Engaging to Learn: Simulations and Games


Adaptation Award

Digital Media Assignments 2009-2012

Survey instruments:
Faculty pre (.pdf)
Faculty post (.pdf)
Students pre (.pdf)
Students post (.pdf)

Technology-Enhanced Collaborative Group Work 2007-2010

Survey instruments:
Faculty pre (.pdf)
Faculty post (.pdf)
Students pre (.pdf)
Students post (.pdf)

2010 Follow-up Survey 1-year out (.pdf)
Evaluation Report (.pdf)
Data Party - Consultant Meeting (Oct. 8, 2009) (.pdf)
Data Party - Engage Faculty Advisory Group Meeting (Jan. 12, 2010) (.pdf)
View the one-pager here (.pdf)

Evaluation and Assessment Presentation (.pdf)

Podcasting & Podcasting Plus 2005-2007

2006-2007: Podcasting Evaluation Report (.pdf)
2008: Podcasting Awardee 1.5 year out Survey Results (.pdf)
2010 Podcasting Awardee 3-year out Survey Results

Technology-Enhanced Collaborative Group Work Follow-up Survey 2010

2011 Follow-up Survey: 2 years out 1-pager
2-year Follow-up Survey Results
The 2008-2009 Engage Adaptation Award on Technology-Enhanced Collaborative Group Work (TECGW) supported campus initiatives including the Strategic Framework, the IT Strategic Plan, LEAP, the Wisconsin Idea and the Wisconsin Experience. As part of the program evaluation a survey was sent to 41 instructors who were involved in the Engage TECGW Adaptation Award. The purpose of the survey was to find out: 1.) Current use of collaborative technologies to support group work, 2.) How, if at all, the award program had an impact on teaching and research, 3.) If the award helped bring in more funding for teaching with technology, 4.) If technology-enhanced group work results are being disseminated to a larger audience. Thirty-five responses were collected for a 85.4% response rate.

  • 65% are still using a collaborative technology for group work
  • 79% indicated the award changed the way they teach
  • 85% have explored other instructional technologies as a result of the award
  • 45% have presented on their experience with Technology-Enhanced Group Work

Podcasting Awardee Follow-up Survey 2010

In November 2010, a brief 3-year follow-up survey was sent to 92 people who were involved in the Podcasting or Podcasting Plus Engage Adaptation Awards. The purpose of the survey was to find out if the past awardees were still using podcasting in their teaching and if they were sharing their experiences with campus and beyond. Fifty-one people responded, for a response rate of 55%.

  • 71% of respondents are still using podcasting in their teaching
  • 40% of respondents have published or presented on their experience using podcasting in their teaching
  • Presentations have been done locally, nationally, and internationally

Technology Enhanced Collaborative Group Work 2008-2009

Evaluation Report
Data Party - Consultant Meeting (Oct. 8, 2009)
Data Party - Engage Faculty Advisory Group Meeting (Jan. 12, 2010)

The 2008 Engage Technology Enhanced Collaborative Group Work (TECGW) Adaptation Award provided 40 instructors with technical and pedagogical support to implement technology enhanced group projects in their courses. The support included a literature review of research related to group work, pedagogical and instructional technology consultation, evaluation consultation, support for technology tools and services, and hosting community events to share challenges and solutions in facilitating group projects.

An evaluation of the program was conducted to gather evidence of methods of good practice in facilitating technology enhanced group projects. The recommendations below, resulting from the instructor, student, and consultant surveys, provide methods of good practice for instructors planning to implement technology enhanced group projects. The results of our award program, in conjunction with the literature on group work, provide evidence of how task design, group process facilitation, and technology choice influence the success of the group project. The recommendations follow.

Be sure the task is worthy of collaborative group work

  • Consider the pedagogical challenge. Will a group add value to the task?
  • Specify how the group project aligns with the course learning objectives.

Design the task

  • Identify the types of interactions desired (student-student, student-instructor, student-content).
  • Identify the tasks students will need to complete.
  • Identify challenges students might face. Where have students had problems in the past?
  • Choose a technology tool that aligns with the interactions, tasks, and challenges.
  • Develop an assessment plan for the process as well as the assigned product.

Prepare students for collaboration

  • Communicate the rationale for group work, including why the challenge is worthy of collaborative group work and how the group project relates to course objectives.
  • Help your students understand what makes good collaboration.

Do regular assessments of process and product

  • Use milestones or check-ins to ensure groups are on the right track.
  • Provide students the opportunity to evaluate their group members’ contributions to the group project.

Podcasting Awardee Follow-up 2008

Survey Results
Participants of the Podcasting Adaptation Award completed a follow-up survey in fall 2008. The purpose of the follow-up survey was to find out 1.) Current use of podcasting for teaching and learning, 2.) How, if at all, the award program had an impact on teaching and research, 3.) If the award helped bring in more funding for teaching with technology, 4.) If podcasting results are being disseminated to a larger audience. Fifty-nine responses were collected for a 59.6% response rate. A brief summary of the results:

  • 74% of the respondents indicated they are still using podcasting in their teaching.
  • 75% indicated they have explored other instructional technologies as a results of the Podcasting Adaptation Award.
  • 79% of respondents indicated the Podcasting Award changed they way the teach.
  • One awardee used the podcasting experience to secure a 5-year, $40,000 grant to develop a new undergraduate course.
  • 29% of respondents have presented on podcasting at various professional conferences, nationally and internationally.
  • 9% of respondents have published articles related to podcasting in teaching.

Podcasting 2006-2007

Evaluation Report
The Podcasting Adaptation Award evaluation included pre- and post-surveys for awardees and students. A list of suggestions from the findings was collected from the surveys to provide methods of good practice to be shared with other instructors wishing to experiment with podcasting. The recommendations are the following:

  • Consult with other instructors and department support staff about their experiences using podcasting.
  • Visit the Engage website for examples of effective podcasts and instructions.
  • Start early, and allow plenty of time to create the podcast episodes.
  • Have a clear idea of why you are incorporating podcasts into your course.
  • Communicate the rationale for using podcasting to your students.
  • Provide students with instructions for accessing the podcasts- don’t assume they already know how to do it.
  • Make listening to the podcasts a required part of the course.
  • Commit to a schedule for developing podcast episodes.
  • Keep the podcasts short.
  • Enjoy it.

Impact Award

Coming soon!