Abstract
Tactile sensing is required for the dexterous manipulation of objects in robotic applications. In particular, the ability to measure and distinguish in real time normal and shear forces is crucial for slip detection and interaction with fragile objects. Here, we report a biomimetic soft electronic skin (e-skin) that is composed of an array of capacitors and capable of measuring and discriminating in real time both normal and tangential forces. It is enabled by a three-dimensional structure that mimics the interlocked dermis-epidermis interface in human skin. Moreover, pyramid microstructures arranged along nature-inspired phyllotaxis spirals resulted in an e-skin with increased sensitivity, minimal hysteresis, excellent cycling stability, and response time in the millisecond range. The e-skin provided sensing feedback for controlling a robot arm in various tasks, illustrating its potential application in robotics with tactile feedback.
INTRODUCTION
Manufacturing robots have been commercially available in industry for more than 50 years (1), and surgical robots are currently revolutionizing health care (2). However, despite the numerous services they can provide, domestic robots are still not part of everyday life. One of the reasons behind this absence is the unmet need for a robotic equivalent to human skin (3) and, more specifically, the lack of capacity for dexterous and in-hand manipulation (tactile sensing on palm and fingertips only).
Methods have been proposed to handle contact on robots by using existing sensors and models (4). However, tactile sensing is required for most manipulation tasks to provide contact parameters such as forces, force direction, contact location, and contact surface. This is particularly relevant for multi-fingered hands with many degrees of freedom, where the internal torque sensing cannot always be implemented because of design or cost constraints. In addition, the ability to measure and discriminate in real time the normal and shear forces is necessary to provide texture and slip information. These parameters cannot be obtained with the traditional wrist force, torque, and proprioceptive sensors currently featured in existing robots (3). Future robots will need this feedback to perform tasks that are trivial for humans, such as holding a glass or inserting a key in a lock. Thus, developing a robotic equivalent of the human skin and its complex sensory system is an inspiring and challenging research topic.
A feature of particular interest in biological skin is the spinosum (Fig. 1A). This microstructure—composed of interlocked hills and localized at the interface between the epidermis and the dermis—produces a localized and high stress concentration at the wrinkle tips near receptors, playing a key role in afferent stimuli for enhanced pressure perception (5). In this area, four types of mechanoreceptors measure innocuous mechanical stimuli on different time scales and with different receptive field sizes, including the slow-adapting receptors (SA-I and SA-II) that respond to static pressures and the fast-adapting receptors (FA-I and FA-II) that respond to dynamic forces and vibrations (Fig. 1A) (5, 6).
(A) Cross-section of the skin from fingertip depicting key sensory structures. The spinosum is a layer found between the dermis and the epidermis, forming interlocked microstructures (hills) responsible for tactile signal amplification (5). Four types of mechanoreceptors measure innocuous mechanical stimuli with different receptive field sizes and time scales (SA-I and SA-II are located near the surface of the skin and deeper, respectively; FA-I and FA-II measure low-frequency and high-frequency stimuli, respectively) (5, 6, 35–38). Because of the 3D structure of the spinosum, the hills concentrate forces onto the mechanoreceptors differently depending on the direction of applied force. (B) Soft biomimetic e-skin. Black, CNT electrodes; blue, PU elastomer; gray, intermediate thin-film dielectric layer, which ensured electrical insulation of the capacitors. During assembly, the bottom and top electrodes were aligned perpendicularly so that an array of capacitors was formed, with each hill corresponding to 25 capacitors (1 capacitor on the top of the hill, 4 on the slopes, 4 in the corners, and 16 surrounding the hill).
Our electronic skin (e-skin) (Fig. 1B) mimics the hills and mechanoreceptors present in the spinosum to detect normal and shear forces. Recent work yielded e-skins capable of detecting sensations similar to human skin, including temperature, pressure, vibration, and strain (6). These e-skins were also inspired by biology in terms of stretchable (7–9), self-healing (10), and biodegradable (11, 12) properties. Recently, several biomimetic strategies have been proposed that took advantage of the interlocked microstructures found in spinosum combined with resistive (contact resistance based) (13–16), piezoresistive (17, 18), ferroelectric (17), triboelectric (19), or capacitive (20) sensor arrays (Fig. 2A) and resulted in multifunctional sensing platforms with improved sensitivity, response time, and linearity. These e-skins were sensitive to various stimuli, including normal force (13–17, 19, 20), shear force (14, 17, 18), lateral strain (stretch) (14), bending strain (13–15, 19, 20), and temperature (17). However, in these platforms, the sensor field has been shown to be uniform across multiple loading conditions, making it difficult to distinguish between them. The distinction between those various stimuli was solely based on the shapes of response curves as a function of time that differed from one mechanical stimulus to the other. This limitation, which was by design to all strategies in Fig. 2A but not to our approach in Fig. 2B, is illustrated in Fig. 2C. The sensor array in Fig. 2C is sensitive to normal force, shear force, the combination of both, and even other stimuli such as bending. It is possible, by looking back at a recorded signal, to evaluate the nature of an unknown stimulus based on the combination of amplitude, shape, and frequency of the signal by referring to a previously known library of stimuli response curves. However, these platforms compromise on robotic hand handling speed and controller performance by design. More precisely, they based their feedback loop on a time series of signals to discriminate between normal and shear forces, which decreased the bandwidth of the closed loop controller because its signal acquisition frequency was divided by the length of the time series needed. For this reason, despite research on tactile sensing being decades old (3–24), the development of robotic hands controlled with tactile feedback has so far focused on the detection of normal forces. Humanoid robots such as PR2 (25) and iCub (26) used arrays of rigid capacitive sensors because the recently developed multifunctional force-sensing platforms still lacked discrimination capabilities. On the other hand, a controller using our proposed sensor can discriminate forces with a spatial signal processing on the grid at each time step so that the controller’s bandwidth (therefore the performance) could be much higher in theory.
(A) Previously proposed concepts for biomimetic e-skins with interlocked microstructures (cross-sectional view), including resistive (contact resistance based) (13–16), piezoresistive (13, 17, 18), ferroelectric (17), triboelectric (19), and capacitive (20) sensing mechanisms. (B) Our proposed concept. The 3D hill structure allows for different deflection capabilities on the top and around of the hills, thus differentiating capacitive responses to a pressure event from different directions. Black lines are side views of electrodes. (C) Schematics showing the limitations associated with previously proposed concepts. R represents the physical variables that are measured in those concepts, for instance, electrical resistance. Although each of the previously proposed concepts used different mechanisms (resistance, piezoresistivity, etc.), they have in common a structure with two flat electrode layers [black lines as in (B)] and an elastic structure (interlocked hills, pyramids, and hairs are gray structures) in between, as illustrated by the cross-sectional views. The pixels in such structures react uniformly across the sensor to applied stimuli regardless of normal (top), shear (middle), or tilt (bottom) force. This is illustrated by the cross-sectional views and the top-view diagrams of the e-skin outputs for arrays of five-by-five sensing pixels, with different orange color intensities for different applied forces. Thus, it is not possible, at a given time, t, and based on the array output, to distinguish the various types of applied forces. (D) Schematics showing the advantages associated with our proposed design. With our design, it is possible to measure and discriminate in real time normal and shear forces and forces applied in various directions. The tilt force is a combination of normal and shear force. In the case of a robotic application, the data of a fraction of the 25 pixels could provide sufficient information (for instance, nine of them—one on the top of the hill, four on the sides, and four in the corners), but the 25 pixels show the significance of our concept.
RESULTS
Here, we propose a soft electronic skin, inspired by human skin (Fig. 1), based on capacitive sensing (Fig. 2B) and its capability of measuring and discriminating in real time both normal and shear forces (Fig. 2D). It consists of an array of capacitors, formed by carbon nanotube (CNT) top and bottom electrodes embedded into a polyurethane (PU) matrix positioned orthogonally to each other (Fig. 1B). An intermediate thin-film dielectric layer ensured electrical insulation of the capacitors. The top layer of the e-skin comprised a grid of molded square pyramids. These microstructures allowed the PU to elastically deform when an external pressure was applied, storing and releasing the energy reversibly, thus minimizing undesirable viscoelastic behavior and resulting in enhanced sensitivity (11, 21). The bottom layer of the e-skin comprised a two-dimensional (2D) array of molded hills, which mimicked the spinosum layer in human skin and were essential for measuring and discriminating the direction of the applied force.
A key aspect of many biological systems is the multiple levels of organization (27). For example, muscles are organized on several different length scales from the cellular to the structural level. This is one of few works, to our knowledge, that use multiple levels of biologically inspired patterning (i.e., hierarchically patterned systems) for tactile sensors. PU elastomer (SG-80A, Tecoflex) was chosen because of its excellent mechanical properties in terms of stretchability and durability, easy processing, and the facile transfer of CNTs on PU. CNT-PU–based electrodes were chosen because of their superior electrical stability upon applied mechanical deformation (28, 29)
Biomimetic e-skin fabrication and characterization
The assembly of the sensor was a benchtop process (Fig. 3A), involving the lamination of the bottom electrode layer with hills, the intermediate dielectric layer, and the top electrode layer with pyramids. Optical and scanning electron microscopy (SEM) images of the hills, pyramids, electrodes, and assembled device are shown in Fig. 3 (B and C), and a schematic of the capacitor structure around a hill is shown in fig. S1. This simple process may be scaled readily and resulted in a geometrical configuration with several key benefits. First, similar to human skin (Fig. 1A), the proposed e-skin presented a high density of mechanoreceptor-like sensors. Each hill corresponded to 25 capacitors each 90,000 μm2 in size (1 capacitor at the top of the hill, 4 on the slopes, 4 on the “corners,” and 16 surrounding the hill), and the location of each sensing pixel was well controlled and ensured by proper alignment.
(A) E-skin fabrication and assembly. The device consists of three layers, assembled by lamination: (i) a bottom 1-mm-thick PU layer with an array of hills (hill diameter, 1 mm; height, 200 μm), (ii) an intermediate 10-μm-thick PHB-PHV dielectric layer used as a spacer between the top and bottom electrodes, and (iii) a top 60-μm-thick PU layer with an array of pyramids. The electrodes were made of spray-coated and photolithography-patterned conducting CNTs embedded into the PU matrix (electrode width, 300 μm; separation distance between two electrodes, 50 μm). The construct was reinforced with tape at the sides, and no sliding of the layers was observed when shear force is applied. For our current sensor size, we observed that the use of tape was sufficient to stabilize the system for laboratory experiments. If we were to scale the sensor array, the proper adhesion between layers would need to be implemented to ensure mechanical stability. (B) Optical image of a fabricated e-skin and close-up view on the hills and electrodes (inset). (C) Optical image showing the CNT-PU interconnects for signal recording with LCR meter and SEM picture of the top e-skin layer with molded pyramids, showing CNT-PU and PU areas (inset).
A key benefit of the proposed design is the ability to detect the direction of applied force, as illustrated with COMSOL simulations (Fig. 2B). Because of the 3D geometry of the hills and the anisotropic deformation of the top layer with applied tilt force, the capacitors located on the side of the hill and exposed to a greater pressure had a larger increase in capacitance than those located on the side opposite the applied force direction. This functionality is additionally illustrated (Fig. 4, A to C) for applied normal force, shear, and tilt force, respectively. The capacitance map around a hill provided the ability to differentiate several types of applied forces, whereas an individual pixel alone was not able to provide this information (more details on mechanism in fig. S2 and pressure-sensing measurement setup in fig. S3). The pressure sensitivity S is defined as the slope of the traces (Fig. 4, A to D), S = δ(ΔC/Cmin)/δP, where C and Cmin are the capacitances with and without applied pressure, and P is the applied pressure. The normal pressure sensitivity for the capacitors located at the top of the hills was 0.19 ± 0.07 kPa−1 in the low-pressure regime (P < 1 kPa), 0.10 ± 0.01 kPa−1 in the range 1 < P < 10 kPa, and 0.04 ± 0.001 kPa−1 in the range 10 < P < 20 kPa. The pressure sensitivity was on average 68% and 30% of these values for the capacitors located on the slope and at the bottom of the hills, respectively. These values are in good agreement with our previous results because the PU used in this study has a similar tensile modulus (~2 MPa) to polydimethylsiloxane (PDMS) (11, 12, 21). They are also in the expected range when compared with other pyramid-based capacitive pressure sensors (15). Moreover, we measured a response time in the millisecond range (fig. S4), which is in good agreement with those previously reported by our group (11).
Pyramids were arranged in a 2D orthogonal grids, the pyramid width was a = 30 μm, and the separation distance between pyramids was b/a = 4 unless stated otherwise. (A to C) Sensor arrays of five-by-five capacitors, centered around one hill, were characterized by measuring the pressure response curves upon applied forces. Top: Cross-sectional views with forces shown in arrows and the locations of the PU-CNT electrodes shown in black. Middle: Top views of the relative changes in capacitance shown for 25 capacitors. Bottom: Measured pressure response curves for three capacitors (one located at the top of the hill and two located at the bottom, surrounding the hill). Dashed circles represent the locations of the hills. (A) Normal force (green arrows) with the relative change in capacitance calculated as ΔC/Cmin = (C700kPa − Cmin)/Cmin, where Cmin and C700kPa are the capacitances without and with applied pressure, respectively. The experiment was reproduced five times. (B) Applied shear force and a necessary normal force of 5 to 10 kPa (red) and ΔC/Cmin with ΔC = (C340kPa − Cmin). The patterns for normal and shear forces are distinct. The experiment was reproduced two times. (C) Applied tilt force (a combination of normal and shear force, dark blue) and ΔC/Cmin with ΔC = (C340kPa − Cmin). The measured pattern combines the characteristics of (A) and (B). The experiment was reproduced two times. (D) Similar to the various types of mechanoreceptors in human skin, capacitors had different pressure response curves (and therefore sensitivities) depending on their locations. Top: Response characteristics, for applied normal force, for two capacitors located at the top and at the bottom of the hills. Bottom: Normalized response curves for the two capacitors. The slopes were used to calculate the sensitivities in various pressure ranges. The experiment was reproduced five times. (E) 3D plots of the measured relative change in capacitance for a sensor array of nine-by-nine capacitors, where each color band corresponds to a variation ΔC/Cmin = 5%. Normal force applied on the entire array (left) and on the bottom left corner (right). The pixels (capacitors) located at the top of the hills have a measured standard deviation on ΔC/Cmin below 20%. Experiments were reproduced two times (left) and one time (right). (F) Device response at applied pressures in the range from 0 to 1800 kPa (normal force, b/a = 2, where a + b is the distance between the centers of two pyramids). The robustness of the sensor is illustrated by the unaltered pressure response curves after several runs at various pressures (inset). The experiment was reproduced two times. (G) The e-skin was sensitive enough to measure objects as small as a 1-mm-diameter plastic bead (15 mg, corresponding to less than 0.5 kPa). The bead was placed on the array (zone 2) and removed several times, and the pressure response signals were measured for capacitors located at the top (left), slope (middle), and bottom (right) of the hill. The experiment was reproduced one time. (H) Cycling test illustrating the stability of the pressure response over 30,000 cycles (b/a = 2). A small signal drift was measured, illustrated by the fact that C15kPa increased by 2.3% and C80kPa increased by 0.2% after 30,000 cycles. The experiment was reproduced two times.
The sensitivity to shear force for the capacitors located at the top and the side of the hills exposed to shear was 3.0 ± 0.5 Pa−1 (10 < P < 20 kPa). The sensitivity of the capacitors located on the side of the hills not exposed to shear was on average 30% that of the exposed side. As with the SA-I receptors in the human sensory system (5, 6), each capacitor, depending on its location on the hill, reacted differently to the same applied force. In addition, as with the human spinosum, the hills concentrated forces onto the receptors differently depending on the direction of applied force (Fig. 4D). At pressures below 70 kPa, the capacitors at the top of the hills had a higher sensitivity than those at the bottom of the hills because of the short capacitor gap and the deformation of the pyramids upon applied pressure. On the other hand, the capacitors located on the side of the hills had a better ability to measure larger forces without reaching saturation: Above 70 kPa, the capacitors surrounding the hills had a pressure sensitivity 9% higher (range from 100 to 600 kPa) than those at the top of the hills because of the deformation of the top membrane. As with biological skin and because of the stretchability of PU, the e-skin enabled the detection of a force exerted on a localized area with limited effect on nearby pixels, as demonstrated with a nine-by-nine sensor array (Fig. 4E). The sensor was designed to work in a range up to a maximum of 100 kPa, a little higher than the typical human touch–sensitive range (~10 kPa). Such a sensor in robotic applications also needs good robustness to high-pressure events (Fig. 4F). For several consecutive runs of pressures 0 to 1800 kPa, the sensor output was highly reproducible, and the characteristic of the device was not altered by the high pressures. Similarly to biological skin, the e-skin was highly sensitive and could detect small weights of 15 mg (Fig. 4G), corresponding to a pressure <0.5 kPa. In this conservative scenario, we calculated a real-time signal-to-noise ratio (SNR) of 3. In most cases, the baseline signal was higher, and we obtained greater SNRs. Moreover, the pressure response of the sensors could be reproducibly cycled thousands of times (Fig. 4H). After applying a pressure of 70 kPa and releasing to 15 kPa for more than 30,000 cycles (duration of 1 cycle: 4 s), the minimum and maximum values of measured capacitance, Cmin and Cmax, increased by 2.3% and 0.2%, respectively.
Biomimetic e-skin design optimization
An optimization of the design of the e-skin was investigated with the objectives to maximize the sensitivity, the SNR, and the time response. For this investigation, we used various pyramid sizes (widths of 10, 20, 30, 40, and 50 μm) and separation distances (ratio b/a = 0.4, 0.8, 1.2, 1.6, 2, and 4, where a + b is the distance between the centers of two pyramids). Zone 1 capacitors were located on the slopes and bottom of the hills, and zone 2 capacitors were at the top of the hills. COMSOL simulations for zone 1 (Fig. 5A) and zone 2 (fig. S6) were performed with the objective to maximize the deflection of the top membrane upon applied pressure. Following the parallel plate capacitor definition of C, this resulted in larger ΔC/Cmin:
(1)where εr is the relative static permittivity, ε0 is the permittivity of vacuum, A is the area of overlap of the two electrodes, and d is the separation between the electrodes. Figure 5B shows that higher top membrane deformation was achieved in zone 1, with smaller pyramids and larger separation distance, resulting in larger ΔC/Cmin and more sensitivity. Moreover, according to Eq. 1, in zone 2, larger Cmin and therefore better SNR were achieved with smaller pyramids (fig. S5A) and smaller separation distance between the pyramids (fig. S5B). In addition, it was previously shown that faster response time could be achieved with smaller separation distance between the pyramids (30).
(A) Optimization of the geometry of the pyramids (size and b/a) for the deflection of the top membrane in zone 1, corresponding to the capacitors located on the slopes and at the bottom of the hills. Top left: Cross-sectional view of the top electrode layer in zone 1 with pyramids, with the geometry used for COMSOL simulation. A 1-kPa uniform pressure was applied (normal force). The initial and deformation patterns are shown in black and red, respectively. Top right: Cross-sectional view showing the e-skin structure with deformed top electrode layer. Bottom: The simulations were performed for a values of 10, 20, 30, 40, and 50 μm and b/a = 0.4, 0.8, 1.2, 1.6, 2, and 4 (a + b is the distance between the centers of two pyramids). (B) Corresponding results for total displacement (left) and stress (right). The distance evaluated for total displacement is indicated with blue arrows. The stress is evaluated at the point indicated with blue circle. The red circle identifies the best conditions (best-case scenario) for high sensitivity. The influence on ΔC/Cmin, when comparing worst-case scenario (a = 50 μm, b/a = 0.4) and best-case scenario (a = 10 μm, b/a = 4, applied pressure of 1 kPa), is ~4%. (C) Summary of the requirements for optimized e-skin, in terms of pyramid width and separation distance, in zones 1 and 2. On the basis of these requirements, the spiral grid gave a good combination of high sensitivity in zone 1 and both high Cmin and fast time response in zone 2. (D) The spatial organization of sunflower florets was used to design the positioning of the pyramids, resulting in enhanced e-skin performances. [Credit: A. Marcus, https://digitalprovocations.wordpress.com/2011/11/10/project-3_2a-sunflower/ (E) Left: Microscope image of the Si masks used to mold the PU top electrode layer, with pyramids arranged along orthogonal grid (top) and spiral grids (bottom). Right: Top electrode layer with pyramids to be positioned on the 1-mm-diameter hills shown at the same scale. (F) SEM image showing the PU top electrode layer with pyramids arranged along phyllotaxis spiral grid. The CNT-PU conducting electrodes appear as light gray stripes (darker gray stripes correspond to PU without CNTs between the electrodes). (G to J) Measured response characteristics of our e-skin, for arrays of five-by-five capacitors with orthogonal and spiral grids of pyramids (experiment reproduced two times each). Top: Cross-sectional view with normal force (green). Middle: ΔC/Cmin shown for the 25 capacitors of each array. Bottom: Measured pressure response curves for all 25 capacitors. (G) Thirty-micrometer-wide pyramids positioned along an orthogonal grid with b/a = 4; (H) same with b/a = 0.4; (I)30-μm-wide pyramids positioned along a phyllotaxis spiral grid (b/a = 0.4 and 4 at the spiral center and border, respectively); (J) same with 10-μm-wide pyramids.
A summary of the requirements for optimized e-skin, in terms of pyramid microstructure design, is provided in Fig. 5C. The question is therefore the following: How should the pyramids be arranged to fulfill the requirements for both zones 1 and 2? A spiral grid gives a good combination of high sensitivity in zone 1 and high Cmin and fast time response in zone 2. This distribution offers a smooth transition of pyramid density from zones 1 to 2—from large to small b/a ratio. Spiral grids directly inspired from botany are the so-called phyllotaxis spirals. They can be seen widely in nature; for example, in the capitulum of sunflower (Fig. 5D), multiple spirals run both clockwise and anticlockwise (31–33). Mathematically, phyllotaxis spirals can be calculated by using the planar model proposed by Vogel (34). This model is based on an analysis of the Fibonacci suite converging at infinity toward the golden number, where every number is the sum of the two preceding ones. The position of each pyramid from the center was therefore defined with the formula
(2)where n is the numbering order of each single pyramid. Only one parameter controls the phyllotaxis pattern, the scaling parameter c (34). On the basis of this formula, new biomimetic e-skins were fabricated where the pyramids were not positioned according to orthogonal grids but rather according to phyllotaxis spiral grids with one spiral per hill (Fig. 5E). The top electrode layer with pyramids was organized along a phyllotaxis spiral grid (Fig. 5F) ready for sensor assembly. The response characteristics of the biomimetic e-skin for sensor arrays of five-by-five capacitors with orthogonal and spiral pyramids grids are shown in Fig. 5 (G to J). Considering zone 1, we measured larger ΔC/Cmin with devices in Fig. 5 (G and I), where the separation distance between the pyramids was large (ratio b/a = 4 versus 0.4 in Fig. 5H). This result is in good agreement with simulations (Fig. 5, A and B). Moreover, Fig. 5 (I and J) shows the response curves for two e-skins with spiral grids, with pyramid widths of 30 and 10 μm, respectively. The difference in color between the center and the edge is less distinct with spiral grids (Fig. 5, I and J) than orthogonal grids (Fig. 5, G and H), corresponding to larger response curves and ΔC/Cmin measured at the border of the hills in the context of spiral grids. In addition, larger Cmin and better SNR were achieved with smaller pyramids (Fig. 5J), as described in fig. S5. These results illustrate the superiority of the spiral grids compared with orthogonal grids.
E-skin used to control a robot arm with normal and shear force feedback
Robotic experiments were performed (Fig. 6) to demonstrate (i) the use of our e-skin to control a robot arm in real time and (ii) that the high sensitivity of the nature-inspired e-skin for normal force and shear force stimuli enabled tasks requiring high dexterity. The limitations of our single-pixel detection experimental setup prevented us from integrating full directional sensing capabilities in the robotic application, which requires developments of a multiplexing acquisition platform. The e-skin was mounted on an artificial hand fixed on a robot arm (Fig. 6A). This design has the advantage of making the mechanical structure of our sensor independent from the movement of the arm; thus, we only measured the contributions of the pressure and shear forces. Figure 6B and fig. S7 show a typical test plate with holes and the preprogrammed consecutive movements executed by the robot arm, unless tactile feedback prevented the execution of the entire downward movement. The e-skin sensor array (red in Fig. 6B, inset) was exposed to either normal (green arrows) or shear (dark red arrows) force. When the e-skin was solely exposed to normal force, the robot arm correctly executed a premature upward movement as soon as the finger touched the test plate at a location with no hole because of sensing of normal force (Fig. 6, C and D). Figure 6E illustrates the second set of experiments. If the e-skin was solely exposed to shear (tangential) force with a light object (ping-pong ball, weight of 2.7 g) placed between the two artificial fingers (Fig. 6E), then shear force (dark red arrow) was exerted on the e-skin as soon as the robot arm went vertically down and the ball pressed on the table. The robot arm correctly interrupted its preprogrammed downward movement as soon as shear force was detected (Fig. 6F) and the ball touched the test plate at a location with no hole (movie S1). Experiments (Fig. 6, G to I) also illustrated the high sensitivity of the e-skin and demonstrated tactile sensing capabilities that allowed the robotic device to interact with deformable and delicate objects, such as a fresh raspberry (movies S2 and S3).
(A) Experimental setup, with the e-skin sensor array mounted on an artificial hand and fixed on a gripper attached to the robot arm. The LCR meter recorded capacitance signals from the e-skin. Data were stored on a server and retrieved by the robot controller to be used in a closed-loop feedback scheme to control the movement of the robot arm in real time. (B) Typical test plate with holes [either eight holes, four holes, or no hole, as illustrated in (D) and (F) below]. The purple arrows show the consecutive movements executed by the robot arm, unless tactile feedback prevented the entire execution of the down movement. Inset: The e-skin, mounted on the artificial finger (in red), was exposed to either normal (green arrows, left) or shear (tangential force, dark red arrows, right) force. (C) Schematic of experiments where the e-skin was solely exposed to normal force, reproduced three times. Purple arrows show the movement executed vertically, unless normal force feedback (green arrow) was detected and prevented the entire execution of downward movement. (D) Experiments were performed with four different test plates with holes. The robot arm went consecutively from positions 1 to 8. For each experiment, the position of the robot arm in the z direction and the measured capacitance C were plotted as a function of time. Red arrows indicate the successful detection of normal force when the finger touched the plate (in the absence of a hole) and the corresponding close-loop control feedback movement upward. (E) Experiments where the e-skin was solely exposed to shear force, reproduced three times. A ping-pong ball (weight, 2.7 g) was positioned between the two artificial fingers. Purple arrows show the movement executed downward, unless shear (tangential) force feedback (dark red arrow) was detected and prevented the entire execution of downward movement. Because of the light weight, only a negligible tangential force contribution due to the weight (gravity) is initially present. (F) Experiments were performed with test plates as described in (D). Red arrows in the measured capacitance plots indicate the successful detection of shear force. When the ping-pong ball touched the table, we observed that it did not slide between the fingers because of the initial small normal force applied to hold it and the surface interaction between PU and the ball. Therefore, the tangential force was measured by the e-skin. This information feedback was used to program the next movement of the robotic arm. In this case, as soon as a tangential force was measured, the robot stopped its downward movement, went back up, and continued the experiment to the next position. (G) The capability to interact with fragile objects was demonstrated with fresh raspberries. The robot arm was preprogrammed to go downward unless normal force was detected. (H) Tactile feedback prevented flattening the raspberry. (I) Without tactile feedback, the fruit was irreversibly crushed.
CONCLUSIONS
We present a biomimetic soft electronic skin composed of an array of capacitors, capable of measuring and discriminating in real time both normal and shear forces. The e-skin was used to control a robot arm in various tasks as a first step toward integration of its high-sensitivity directional sensing capabilities, illustrating its potential future application in various fields of robotics, including personalized domestic help, ambulatory and inpatient health care, medical diagnosis, surgery, industry, and exploratory missions in hard-to-reach places.
MATERIALS AND METHODS
Study design
In our experiments (Figs. 4 to 6), data collection rules had their basis in pressure ranges predefined before each experiment. The pressure gauge (fig. S3) or the robotic hand controller (Fig. 6) automatically reversed upon detection of a pressure maxima. Outliers were identified by analyzing the SNR of the response curve, which allowed for identification of faulty sensor fabrication or bad connectivity at the interfaces. In those cases, the sensor was eliminated from the test batch, the two electrode layers were realigned, or connectivity with the LCR meter (inductance-capacitance-resistance) was adjusted. Experiments were then reproduced with the new sensor. Each experiment was reproduced a different number of times, which is indicated in the corresponding figure captions.
Device fabrication
Fabrication of the silicon (Si) wafer with pyramid grids
Si wafers were patterned with pyramids of different sizes by lithography followed by HF oxide etching and finally an anisotropic potassium hydroxide (KOH) etching.
Patterning of the CNT electrodes on PU substrate
This fabrication step applies to the top and bottom PU electrodes. Si wafers were cleaned with O2 plasma. A CNT layer was spray coated on the wafers from a CNT dispersion (12 mg of P2-SWNT from Carbon Solutions and 70 ml of N-methyl-2-pyrrolidone ultrasonicated for 30 min, followed by collecting the supernatant after centrifugation for 30 min at 8000 rpm, 18°C). The CNT electrodes were lithographically patterned by using S1813 photoresist. For the top electrode with pyramid grid, the photolithography mask was aligned with the pyramids to ensure a proper positioning of the CNT electrodes. A subsequent oxygen plasma etching was used to remove the CNTs without photoresist protection. The remaining photoresist was then removed by using acetone, isopropanol, and water. On this wafer, PU elastomer (Tecoflex SG-80A from Lubrizol Co.) was cast from chloroform solution (10 mg/ml) by spin coating at 1000 rpm, followed by another layer of PU from chloroform solution (60 mg/ml) at 1000 rpm. The first layer was used to promote adhesion to CNTs, whereas the thicker layer (~10 μm) allowed easy manipulation of the electrodes. The PU films with CNT-patterned electrodes were then released from the glass substrate for sensor assembly.
Fabrication of the hill arrays
The CNT-patterned PU electrode film (without pyramids) was placed on a grid with 1-mm holes, with the CNT lines aligned with the grid (three lines per hole, one line in between each hole, as shown in Figs. 1B and 3B, insets). Vacuum (~250 torr) was applied to create the hill shape in PU. A PDMS layer (ratio, 1:10; thickness, ~3 mm; PDMS Sylgard from Dow Corning Co.) was cast onto the electrode film and then oven baked for 30 min at 80°C. The final bottom electrode with hills was then released and ready for assembly.
Sensor assembly
The sensor was assembled by laminating the bottom electrode with hills, the 10-μm-thick polyhydroxybutyrate-polyhydroxyvalerate (PHB-PHV) dielectric layer, and the top electrode with pyramids. During lamination, the two electrodes were aligned perpendicular to each other so that each hill corresponded to 25 capacitors (1 on the top of the hill, 4 on the slopes, 4 on the corners, and 16 shared capacitors surrounding the hill, as shown in fig. S1). The alignment was made manually by using an optical microscope. Moreover, PHB-PHV was used in our previous works (11) and was chosen as dielectric because it combines good mechanical resistance at low thickness and moderate dielectric constant to maximize for capacitance, as described in Eq. 1.
Device characterization
Force response measurement setup
The measurement setup consisted of a motorized vertical stage used in combination with a force gauge, and the capacitance of each sensor was measured with an LCR meter (further described in fig. S3).
Robot arm setup
The e-skin was fixed on a mock-up flexible hand, which was attached to a Schunk WSG 50 gripper mounted on a robot arm (KUKA IIWA). The robot was programmed to perform a series of predefined movements, as defined by the experimental protocol, with a controller that could stop the movement depending on the signal recorded on the e-skin. The control algorithm took as input the signal from the e-skin through the LCR and stopped the movement of the robotic arm if the signal reached a predefined capacitance threshold. A redis interface was used for the communication between the LCR and the computer controlling the robot.
SUPPLEMENTARY MATERIALS
robotics.sciencemag.org/cgi/content/full/3/24/eaau6914/DC1
Fig. S1. Schematics of the position of the different capacitors or “pixels” around a hill.
Fig. S2. Measured response characteristics of the biomimetic e-skin.
Fig. S3. Experimental setup used to characterize the e-skin.
Fig. S4. Response time of sensor.
Fig. S5. Optimization of the separation distance d between the top and bottom electrodes of the capacitors in zone 2 (capacitors located at the top of the hills).
Fig. S6. Optimization of the e-skin regarding the time response (zone 2, capacitors located at the top of the hills).
Fig. S7. Experiments with e-skin mounted on a robot arm.
Movie S1. The robot arm correctly interrupts its preprogrammed movement in downward direction as soon as shear force is detected and the ball touches the test plate at a location with no hole.
Movie S2. The high sensitivity of the e-skin allows for interaction with fragile, deformable, and delicate objects such as a fresh raspberry (when tactile sensing is activated, the e-skin senses the contact with raspberry and the robot arm moves in upward direction without damaging the fruit).
Movie S3. When tactile sensing is not activated, the fruit is crushed.
This is an article distributed under the terms of the Science Journals Default License.
REFERENCES AND NOTES
- Copyright © 2018 The Authors, some rights reserved; exclusive licensee American Association for the Advancement of Science. No claim to original U.S. Government Works
