Defining and Evaluating Argumentation Quality in the Context of Design Thinking: Using High School Students’ Design Critiques from Foundational Engineering Courses

Abstract

This research investigates students’ argumentation quality in engineering design thinking. We implemented Learning by Evaluating (LbE) using Adaptive Comparative Judgment (ACJ), where students assess pairs of items to determine the superior one. In ACJ, students provided rationales for their critiques, explaining their selections. Fifteen students participated in an LbE exercise before starting their backpack design projects, critically evaluating multiple backpack designs and producing 145 comments. Writing comments required students to discern and justify the superior design, fostering informed judgment and articulation of their reasoning. The study used the Claim, Evidence, and Reasoning (CER) framework, adapted for engineering design thinking, to analyse these critiques. The framework emphasized three aspects: Empathy (understanding user needs), Ideation (deriving design inspiration), and Insight (gaining valuable understanding from evaluated designs). We employed both deductive and inductive content analysis to evaluate the argumentation quality in students’ critiques. High-quality argumentation was identified based on six codes: user-focused empathy, design inspirations, logical rationalizations, multi-criteria evaluations, aesthetic considerations, and cultural awareness. Poor-quality argumentation lacked these elements and was characterized by vagueness, uncertainty, brevity, inappropriateness, irrelevance, gender bias, and cultural stereotyping. By identifying critical elements of effective argumentation and common challenges students may face, this study aims to enhance argumentation skills in engineering design thinking at the secondary education level. These insights are intended to help educators prepare students for insightful and successful argumentation in engineering design projects.