d.tools: Enabling rapid prototyping for physical interaction design

Project Overview
What kind of tools would you need to make a functional interactive prototype of a media player in 30 minutes?

d.tools is a hardware and software system that enables designers to rapidly prototype the bits (the form) and the atoms (the interaction model) of physical user interfaces in concert. d.tools was built to support design thinking rather than implementation tinkering. With d.tools, designers place physical controllers (e.g., buttons, sliders), sensors (e.g., accelerometers), and output devices (e.g., LEDs, LCD screens) directly onto form prototypes, and author their behavior visually in our software workbench.

The d.tools architecure was inspired by fieldwork conducted at Bay Area design studios and Stanford's graduate product design program. After building an initial prototype with Flash and Phidgets, d.tools has now turned into a full-fledged plug-in for Eclipse with its own dedicated hardware platform. D.tools has been used to re-create existing devices, is currently being deployed at a professional product design consultancy, and will be given to students in Stanford's HCI Design Studio course this winter.

The d.tools Editor
Our visual authoring environment is inspired by the Statecharts visual language and written in Java as a plug-in for the Eclipse IDE. States in the editor specify device outputs; state transitions are triggered by physical inputs. Users graphically arrange icons of rec-ognized physical I/O components into a virtual representa-tion of the physical device. This iconic representation affords rapid matching of software widgets with physical I/O components. d.tools dynamically detects the presence and capabilities of attached hardware components, enabling the software editor to be coupled to the hardware configuration.
The hardware platform

d.tools provides plug-and-play prototyping of hardware components by making each component smart (adding a dedicated microcontroller) and networking the components on a common bus. For graphics display on the small screens commonly found in information appliances, d.tools includes an LCD display which can be connected to a PC graphics card with video output. To facilitate extensibility by advanced users and the software development community at large, d.tools builds on existing open source APIs for its wire protocol (I2C) and the hardware-to-PC interface (OpenSoundControl).

NEW (07/2006): You can now connect the d.tools software environment to other commercially available hardware platforms. Please see the new page on support for Wiring boards, Arduino boards and Phidgets InterfaceKits.


Demonstration of design-test-analyze support in d.tools, April 2006
H.264 Quicktime version (MOV, 1280x720, 4:30, 40MB).

UIST 2006 Talk, October 2006 (20 minutes, different formats available)


Hartmann, B., Klemmer, S.R., Bernstein, M., Abdulla, L., Burr, B., Robinson-Mosher, A., Gee, J.
Reflective physical prototyping through integrated design, test, and analysis.
Proceedings of UIST 2006, October 2006 - Best paper award!

Scott R. Klemmer, Bjoern Hartmann, and Leila Takayama
How Bodies Matter: Five Themes for Interaction Design.
Proceedings of DIS2006, June 2006.
Talk slides (ppt), presented on October 18, 2006, at UIST in Montreux.

All d.tools code is available for free under the open source BSD license. You can download the latest packaged binary release of d.tools from our our sourceforge.net project page. Access to source code is available through anonymous CVS via d-tools.cvs.sourceforge.net, repository path /cvsroot/d-tools/ (see this Sourceforge CVS page for configuration info)

Installation instructions for the d.tools Visual Authoring Environment
User guide for the d.tools Visual Authoring Environment
Developer guide (javadoc)
d.tools lite (Java API for working with d.tools hardware without the visual editor)
Hardware resources (updated 2006-05-26)

Related Projects

Exemplar is a tool that introduces new techniques for authoring sensor based interactions through programming by demonstration.

Björn Hartmann
Scott Klemmer
Leith Abdulla
d.tools alumni: Nirav Mehta, Michael Bernstein, Brandon Burr, Avi Lev Robinson-Mosher, Jennifer Gee