Understanding How Prototyping Practices Affect Design Results
Prototypes created during design experiments.          Prototypes created during design experiments.
Summary
How do prototyping practices affect learning, motivation, communication, and outcome in design? This research examines aspects of the creative process such iteration and comparison, two key strategies for discovering contextual design variables and their interrelationships. We found that, even under tight time constraints when the common intuition is to stop iterating and start refining, iterative prototyping helps designers learn. Our experiments also indicate that creating and receiving feedback on multiple prototypes in parallel – as opposed to serially – leads to more divergent ideation, more explicit comparison, less investment in a single concept, and better overall design performance. Most recently, we found that groups who produce and share multiple prototypes report a greater increase in rapport, exchange more verbal information, share more features, and overall, reach a better consensus.

What's our approach? We recruit people to participate in tasks where the solutions are creatively diverse and objectively measurable. We've had success using the egg drop design task where participants create a vessel to protect a raw egg. Most recently, our studies call on participants to create Web banner ads. We then place the ads in an online campaign, collect a host of analytics (e.g., click-through data), and statistically compare performance differences. The key insight enabling our research is that crowdsourcing techniques and web analytics provide an opportunity to do experimental research on creativity with objective outcomes.

overview talk
Publications
Early and Repeated Exposure to Examples Improves Creative WorkChinmay Kulkarni, Steven P Dow, Scott R Klemmer. Cognitive Science, 2012    SLIDES
Shepherding the Crowd Yields Better WorkSteven P. Dow, Anand Kulkarni, Scott R. Klemmer, Bjoern Hartmann. CSCW: ACM Conference on Computer Supported Cooperative Work, 2012    SLIDES
Investigates how feedback affects crowdsourced work. A field experiment with the Shepherd system shows that self-assessment and external feedback help workers learn task criteria and produce better work.
Prototyping Dynamics: Sharing Multiple Designs Improves Exploration, Group Rapport, and ResultsSteven P Dow, Julie Fortuna, Dan Schwartz, Beth Altringer, Daniel L Schwartz, and Scott R Klemmer. CHI: ACM Conference on Human Factors in Computing Systems, 2011    SLIDES
Creating and sharing multiple alternatives with peers leads to more individual exploration, better integration of others' ideas, more productive design conversations, and higher-rated, better performing design results.
Parallel Prototyping Leads to Better Design Results, More Divergence, and Increased Self-EfficacySteven P Dow, Alana Glassco, Jonathan Kass, Melissa Schwarz, Daniel Schwartz, Scott R Klemmer. ACM Transactions on Computer-Human Interaction, 2010    SLIDES
Creating multiple designs and receiving feedback in parallel--rather than serially--yields better, more diverse ideas, more comparison, and less fixation. (Measures: human raters, online analytics.)
The Efficacy of Prototyping Under Time ConstraintsSteven P. Dow, Kate Heddleston, Scott R. Klemmer. Creativity & Cognition, 2009
People

Steven Dow, spdow at stanford.edu
Scott Klemmer, srk at stanford.edu
Dan Schwartz, danls at stanford.edu


Our research is made possible by the generous financial support of the National Science Foundation, Hasso Plattner Research Program, and Media X.