Abstract

Designers routinely create informal “thinking” sketches to explore a design space, “talking” sketches to communicate design ideas during the early phases of the design process, and “learning” prototypes to test potential concepts. This study presents two new tools to assess novice designers’ sketch attributes and prototyping reflections in the context of an introductory design course. First, it proposes a rubric for assessing the quality of early-stage design sketches including line smoothness, proportion, and understandability. Of particular note is the contribution of assessing understandability as a metric for sketches as communication tools. This study also presents a tool to capture designer reflections after each iteration of a prototype. Not only does this record what is learned about a design but also designers’ personal and emotional reactions to the process. Sketching-related results show a positive correlation between sketch quality and understandability, indicating the importance of sketch quality especially when designers use sketches to communicate. Results also indicate that early-stage sketch quantity, but not quality, is linked with design outcomes. This study also finds a link between the frequency of sketching and higher maximum sketch quality scores (i.e., at least one highly rated sketch) as well as a correlation between individuals’ maximum sketch quality scores and overall design outcomes. Preliminary results around prototyping indicate that reflection on both the technical and emotional aspects of prototyping may be a worthwhile area of further study. Finally, several results point to novice designers’ lack of consistent focus on users in their prototyping reflections and presentations.

References

1.
Kudrowitz
,
B. M.
, and
Wallace
,
D.
,
2013
, “
Assessing the Quality of Ideas From Prolific, Early-Stage Product Ideation
,”
J. Eng. Des.
,
24
(
2
), pp.
120
139
.
2.
Yang
,
M. C.
,
2009
, “
Observations on Concept Generation and Sketching in Engineering Design
,”
Res. Eng. Des.
,
20
(
1
), pp.
1
11
.
3.
Häggman
,
A.
,
Tsai
,
G.
,
Elsen
,
C.
,
Honda
,
T.
, and
Yang
,
M. C.
,
2015
, “
Connections Between the Design Tool, Design Attributes, and User Preferences in Early Stage Design
,”
ASME J. Mech. Des.
,
137
(
7
), p.
071408
.
4.
Kudrowitz
,
B.
,
Te
,
P.
, and
Wallace
,
D.
,
2012
, “
The Influence of Sketch Quality on Perception of Product-Idea Creativity
,”
Artif. Intell. Eng. Des. Anal. Manuf.
,
26
(
3
), pp.
267
279
.
5.
Hilton
,
E.
,
Willifor
,
B.
,
Li
,
W.
,
McTigue
,
E.
,
Hammond
,
T.
, and
Linsey
,
J.
,
2016
, “
Consistently Evaluating Sketching Ability in Engineering Curriculum
,”
Fourth International Conference on Design Creativity
,
Georgia Institute of Technology, Atlanta, GA
,
Nov. 2–4
.
6.
Rodgers
,
P. A.
,
Green
,
G.
, and
McGown
,
A.
,
2000
, “
Using Concept Sketches to Track Design Progress
,”
Des. Stud.
,
21
(
5
), pp.
451
464
.
7.
Sevier
,
D. C.
,
Jablokow
,
K.
,
McKilligan
,
S.
,
Daly
,
S. R.
,
Baker
,
I. N.
, and
Silk
,
E. M.
,
2017
, “
Towards the Development of an Elaboration Metric for Concept Sketches
,”
ASME 2017 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference
,
Cleveland, OH
,
Aug. 6–9
, p.
V003T04A006
.
8.
Goel
,
V.
,
1995
,
Sketches of Thought
,
MIT Press
,
Cambridge, MA
.
9.
Hammond
,
T.
,
Kumar
,
S. P. A.
,
Runyon
,
M.
,
Cherian
,
J.
,
Williford
,
B.
,
Keshavabhotla
,
S.
,
Valentine
,
S.
,
Li
,
W.
, and
Linsey
,
J.
,
2018
, “
It’s Not Just About Accuracy: Metrics That Matter When Modeling Expert Sketching Ability
,”
ACM Trans.Interact Intell. Syst. Tiis.
,
8
(
3
), p.
19
.
10.
Ferguson
,
E. S.
,
1994
,
Engineering and the Mind’s Eye
,
MIT Press
,
Cambridge, MA
.
11.
Hilton
,
E. C.
,
Gamble
,
T.
,
Li
,
W.
,
Hammond
,
T.
, and
Linsey
,
J. S.
,
2018
, “
Back to Basics: Sketching, Not CAD, Is the Key to Improving Essential Engineering Design Skills
,”
Volume 7 30th International Conference on Design Theory and Methodology
, p.
V007T06A051
.
12.
Nelson
,
J.
,
Berlin
,
A.
, and
Menold
,
J.
,
2019
, “
ARCHIE: An Automated Data Collection Method for Physical Prototyping Efforts in Authentic Design Situations
,” p. V007T06A044.
13.
Lauff
,
C.
,
Kotys-Schwartz
,
D.
, and
Rentschler
,
M. E.
,
2017
, “
Perceptions of Prototypes: Pilot Study Comparing Students and Professionals
,” p.
V003T04A011
.
14.
Schön
,
D.
,
1983
,
The Reflective Practitioner: How Professionals Think in Action
,
Temple Smith
,
London
.
15.
Turns
,
J.
,
Sattler
,
B.
,
Yasuhara
,
K.
,
Borgford-Parnell
,
J.
, and
Atman
,
C.
,
2014
, “
Integrating Reflection into Engineering Education
,” pp.
24.776.1
24.776.16
.
16.
Adams
,
R. S.
,
Turns
,
J.
, and
Atman
,
C. J.
,
2003
, “
Educating Effective Engineering Designers: The Role of Reflective Practice
,”
Des. Stud.
,
24
(
3
), pp.
275
294
.
17.
Deininger
,
M.
,
Daly
,
S. R.
,
Sienko
,
K. H.
, and
Lee
,
J. C.
,
2017
, “
Novice Designers’ Use of Prototypes in Engineering Design
,”
Des. Stud.
,
51
, pp.
25
65
.
18.
Gerber
,
E.
, and
Carroll
,
M.
,
2012
, “
The Psychological Experience of Prototyping
,”
Des. Stud.
,
33
(
1
), pp.
64
84
.
19.
Bamberger
,
J.
, and
Schön
,
D. A.
,
1983
, “
Learning as Reflective Conversation With Materials: Notes From Work in Progress
,”
Art Educ.
,
36
(
2
), p.
68
.
20.
Bao
,
Q.
,
Faas
,
D.
, and
Yang
,
M.
,
2018
, “
Interplay of Sketching & Prototyping in Early Stage Product Design
,”
Int. J. Des. Creativity Innov.
,
4
(
3
), pp.
146
168
.
21.
Robertson
,
B. F.
, and
Radcliffe
,
D. F.
,
2009
, “
Impact of CAD Tools on Creative Problem Solving in Engineering Design
,”
Comput. Aided Des.
,
41
(
3
), pp.
136
146
.
22.
Kwon
,
J.
, and
Kudrowitz
,
B.
,
2019
, “
The Sketch Quality Bias: Evaluating Descriptions of Product Ideas With and Without Visuals
,”
International Design Engineering Technical Conferences and Computers and Information in Engineering Conference
,
Anaheim, CA
,
Aug. 18–21
, American Society of Mechanical Engineers, vol. 59278, p.
V007T06A005
.
23.
Neeley
,
W. L.
,
Lim
,
K.
,
Zhu
,
A.
, and
Yang
,
M. C.
,
2013
, “
Building Fast to Think Faster: Exploiting Rapid Prototyping to Accelerate Ideation During Early Stage Design
,”
American Society of Mechanical Engineers
, p.
V005T06A022
.
24.
Häggman
,
A.
,
Honda
,
T.
, and
Yang
,
M. C.
,
2013
, “
The Influence of Timing in Exploratory Prototyping and Other Activities in Design Projects
,” p.
V005T06A023
.
25.
Yang
,
M. C.
,
2005
, “
A Study of Prototypes, Design Activity, and Design Outcome
,”
Des. Stud.
,
26
(
6
), pp.
649
669
.
26.
Lauff
,
C.
,
Menold
,
J.
, and
Wood
,
K. L.
,
2019
, “
Prototyping Canvas: Design Tool for Planning Purposeful Prototypes
,”
Proc. Des. Soc. Int. Conf. Eng. Des.
,
1
(
1
), pp.
1563
1572
.
27.
Menold
,
J.
,
Jablokow
,
K.
, and
Simpson
,
T.
,
2017
, “
Prototype for X (PFX): A Holistic Framework for Structuring Prototyping Methods to Support Engineering Design
,”
Des. Stud.
,
50
, pp.
70
112
.
28.
Menold
,
J.
,
Simpson
,
T. W.
, and
Jablokow
,
K. W.
,
2016
, “
The Prototype for X (PFX) Framework: Assessing the Impact of PFX on Desirability, Feasibility, and Viability of End Designs
,” p.
V007T06A040
.
29.
Li
,
Y.
, and
Xu
,
W.
,
2021
, “
Using CycleGAN to Achieve the Sketch Recognition Process of Sketch-Based Modeling
,” pp.
26
34
.
30.
Ulrich
,
K. T.
,
Eppinger
,
S. D.
, and
Yang
,
M. C.
,
2020
,
Product Design and Development
,
McGraw-Hill Education
,
New York
.
31.
Hansen
,
C. A.
,
Jensen
,
L. S.
,
Özkil
,
A. G.
, and
Pacheco
,
N. M. M.
,
2020
, “
Fostering Prototyping Mindsets in Novice Designers With the Prototyping Planner
,” pp.
1725
1734
.
32.
Tiong
,
E.
,
Seow
,
O.
,
Teo
,
K.
,
Silva
,
A.
,
Wood
,
K. L.
,
Jensen
,
D. D.
, and
Yang
,
M. C.
,
2018
, “
The Economies and Dimensionality of Prototyping: Value, Time, Cost and Fidelity
,” p.
V007T06A045
.
33.
Menold
,
J.
,
Berdanier
,
C.
,
McComb
,
C.
,
Hocker
,
E.
, and
Gardner
,
L.
,
2018
, “
‘Thus, I Had to Go With What I Had’: A Multiple Methods Exploration of Novice Designers’ Articulation of Prototyping Decisions
,” p.
V007T06A047
.
34.
Lauff
,
C.
,
Kotys-Schwartz
,
D.
, and
Rentschler
,
M. E.
,
2018
, “
What Is a Prototype? What Are the Roles of Prototypes in Companies?
,”
ASME J. Mech. Des.
,
140
(
6
), p.
061102
.
35.
Freelon
,
D. G.
,
2013
, “
ReCal OIR: Ordinal, Interval, and Ratio Intercoder Reliability as a Web Service
,”
Int. J.f Intern. Sci.
,
1
(
8
), pp.
10
16
.
36.
Freelon
,
D. G.
,
2010
, “
ReCal: Intercoder Reliability Calculation as a Web Service
.”
37.
Landis
,
J. R.
, and
Koch
,
G. G.
,
1977
, “
The Measurement of Observer Agreement for Categorical Data
,”
Biometrics
,
33
(
1
), pp.
159
174
.
38.
Mohedas
,
I.
,
Daly
,
S. R.
, and
Sienko
,
K. H.
,
2015
, “
Requirements Development: Approaches and Behaviors of Novice Designers
,”
ASME J. Mech. Des.
,
137
(
7
), p.
071407
.
39.
Génova
,
G.
,
Fuentes
,
J. M.
,
Llorens
,
J.
,
Hurtado
,
O.
, and
Moreno
,
V.
,
2013
, “
A Framework to Measure and Improve the Quality of Textual Requirements
,”
Requir. Eng.
,
18
(
1
), pp.
25
41
.
40.
Nelson
,
J.
, and
Menold
,
J.
,
2020
, “
Opening the Black Box: Developing Metrics to Assess the Cognitive Processes of Prototyping
,”
Des. Stud.
,
70
, p.
100964
.
You do not currently have access to this content.