Identify Design Problems Through Questioning: Exploring Role-playing Interactions with Large Language Models to Foster Design Questioning Skills
Companion Publication of the 2024 Conference on Computer-Supported Cooperative Work and Social Computing, 2024 (CSCW 2024 Poster)
Abstract
Identifying design problems is a crucial step for creating plausible solutions, but it is challenging for design novices due to their limited knowledge and experience. Questioning is a promising skill that enables students to independently identify design problems without being passive or relying on instructors. This study explores role-playing interactions with Large Language Model (LLM)-powered Conversational Agents (CAs) to foster the questioning skills of novice design students. We proposed an LLM-powered CA prototype and conducted a preliminary study with 16 novice design students engaged in a real-world design class to observe the interactions between students and the LLM-powered CAs. Our findings indicate that while the CAs stimulated questioning and reduced pressure to ask questions, it also inadvertently led to over-reliance on LLM responses. We proposed design considerations and future works for LLM-powered CA to foster questioning skills.
BibTeX
@inproceedings{10.1145/3678884.3681912,
author = {Lim, Hyunseung and Choi, Dasom and Hong, Hwajung},
title = {Identify Design Problems Through Questioning: Exploring Role-playing Interactions with Large Language Models to Foster Design Questioning Skills},
year = {2024},
isbn = {9798400711145},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
url = {https://doi.org/10.1145/3678884.3681912},
doi = {10.1145/3678884.3681912},
abstract = {Identifying design problems is a crucial step for creating plausible solutions, but it is challenging for design novices due to their limited knowledge and experience. Questioning is a promising skill that enables students to independently identify design problems without being passive or relying on instructors. This study explores role-playing interactions with Large Language Model (LLM)-powered Conversational Agents (CAs) to foster the questioning skills of novice design students. We proposed an LLM-powered CA prototype and conducted a preliminary study with 16 novice design students engaged in a real-world design class to observe the interactions between students and the LLM-powered CAs. Our findings indicate that while the CAs stimulated questioning and reduced pressure to ask questions, it also inadvertently led to over-reliance on LLM responses. We proposed design considerations and future works for LLM-powered CA to foster questioning skills.},
booktitle = {Companion Publication of the 2024 Conference on Computer-Supported Cooperative Work and Social Computing},
pages = {598–602},
numpages = {5},
keywords = {conversational agents, design education, large language model, questioning},
location = {San Jose, Costa Rica},
series = {CSCW Companion '24}
}