(F-240) Impact of quiz format on student learning and experience in a technical writing course for bioengineers

Technical Writing for Mechanical Engineers and Bioengineers is a writing-intensive sophomore level course taught in a flipped format. Students watch videos on each week’s content prior to class meetings, take a short quiz to assess their understanding, and spend class time on high impact writing practice and discussion. While the quiz has historically been given during class, during periods of remote learning quizzes were often administered online and due before the beginning of class. This convenient format freed up class time for more hands-on work at the cost of immediate feedback and in-class discussion of quiz content. While both in class (IC) and take-home or out-of-class (OOC) assessments have identified advantages and disadvantages, there is not a clear consensus in the literature regarding which assessment format best supports student learning and material retention or how these formats might impact student experiences and perspectives on a course, particularly in the flipped technical writing classroom. To investigate the effect of quiz format, students were thus assigned to take IC or OOC quizzes and their quiz performance, material retention, and engagement with the course were evaluated.

56 students enrolled in 5 sections of the course were assigned to receive quizzes in either IC or OOC format. Quizzes were identical across sections and consisted of ~10 multiple choice questions on the week’s video content. Student scores on these 13 weekly quizzes were recorded throughout the semester as a measurement of how well students understood the lesson content in the provided videos. At the end of the semester, students were additionally given a retention quiz designed to assess both direct recall of familiar content and student ability to apply their knowledge to new problems. Student questionnaires administered at the end of the semester assessed student perspectives on the course, quiz format, and their own learning. The questionnaire included 11 Likert scale questions on student impressions of quiz efficacy and their own study habits, 17 Likert scale questions that asked students to rate their level of confidence in their ability to perform key tasks related to course learning objectives, and 3 free response questions to gather student thoughts on advantages and disadvantages of the quiz format they experienced.

Quiz format was found to have only subtle effects on student quiz scores, with OOC quizzes having slightly higher scores (Figure 1A). Students from the IC quiz group noted that they often forgot video content by quiz time (24.1% of responses) and found quizzes stressful (10.3%), while students from the OOC quiz group appreciated that quizzes could be taken at their convenience (50%) and were less stressful (13.6%). Taking quizzes in a more comfortable environment may have helped improve student quiz performance, but it did not substantially improve retention. On an end-of-semester retention quiz, IC students had only subtly higher scores on “old” questions taken verbatim from past quizzes (Figure 1B), potentially due to the in-class feedback and discussion they received. However, IC and OOC groups performed identically on new questions on the same content (Figure 1C), suggesting that quiz format did not impact students’ ability to apply their knowledge to new situations.

Quiz format also impacted student interaction with the course. When asked how their quiz format impacted their attendance, OOC students had an average score of 3 (no effect), while IC students had a score of 3.9 (more likely to attend) (Figure 2A). This effect was particularly pronounced in students from groups often underrepresented in engineering (Figure 2B), including women, BIPOC, and first-generation students. Additionally, female students who took OOC quizzes felt that the flipped class format was slightly less effective than those who took IC quizzes (Figure 2C). In spite of these student-identified advantages of IC quizzes, students preferred OOC quizzes (66%) to IC quizzes (34%), citing that OOC quizzes were convenient, less stressful, helped them prepare for class, and allowed more in-class time for other work.

In conclusion, both quiz formats had advantages. IC quizzes improved student motivation to attend class, which is important in flipped courses in which attendance and participation are vital to course success. OOC quizzes, on the other hand, were preferred by students, slightly increased quiz scores, and freed up class time for active practice. Neither quiz format significantly impacted retention, making both formats viable in future semesters.