FIELD

The invention relates to a manufacturing method for a tactile sensor, and more particularly to a manufacturing method for a tactile sensor of the type that is used in a robotic end effector such as a robotic hand.

BACKGROUND

In the field of robotics, it is often necessary to determine when an object has been contacted by an end effector or a similar device. For example, if an object is to be picked up by an end effector at the end of a robotic arm, it is important to ensure that the object is correctly located relative to the end effector, and this can be determined when the end effector makes contact with the object. Similarly, the alignment of the object with the end effector should also be determined when the end effector makes contact with the object. Such sensing is particularly important if the end effector is relatively delicate, such as a robotic hand.

A number of types of tactile sensor are already known. For example, the “TacTip” sensor, developed at Bristol Robotics Laboratory, includes a flexible curved surface, on the inner (concave) surface of which are provided a number of pins (or papillae). A camera captures an image of the inner ends of the pins. When the surface is deformed by contact with an object, the inner ends of the pins move, and this movement can be seen by the camera. However, forming the curved surface with the pins is not straightforward; 3D printing is possible, but 3D printed materials are not particularly robust. Further, a considerable depth is needed to accommodate the pins, so the sensor has a minimum size, and may not be suitable for more delicate applications.

The “GelSight” sensor, developed at MIT’s Computer Science and Artificial Intelligence Laboratory, uses a block of transparent rubber with a contact surface coated with a metallic paint. When the painted surface is pressed against an object, it conforms to the shape of the object. The side of the block opposite the contact surface is illuminated by three differently-coloured lights, and imaged by a camera. The camera captures images of the deformed surface in three different colours, and uses these to determine the shape of the object. Although this type of sensor gives a good image of the object, it does not provide a good representation of the tangential or normal forces involved in the contact, and does not allow small surface vibrations to be measured and localized. Further, since the metallic paint is exposed and contacted by the object, the sensor is vulnerable to wear. The sensor is also quite large, and again may not be suitable for more delicate applications. SUMMARY

It is an object of the present invention to provide a method of manufacturing a tactile sensor which goes some way to overcoming the abovementioned disadvantages or which at least provides the public or industry with a useful choice.

The term “comprising” as used in this specification and indicative independent claims means “consisting at least in part of”. When interpreting each statement in this specification and indicative independent claims that includes the term “comprising”, features other than that or those prefaced by the term may also be present. Related terms such as “comprise” and “comprises” are to be interpreted in the same manner.

As used herein the term “and/or” means “and” or “or”, or both.

As used herein “(s)” following a noun means the plural and/or singular forms of the noun.

Accordingly, in a first aspect the present invention may broadly be said to consist in a method of manufacturing a tactile sensor comprising the steps of: i) choosing a flexible material suitable to produce a flexible layer that forms at least part of the sensor; ii) arranging the flexible material so that it forms a substantially continuous layer or sheet; iii) preparing a punch apparatus that comprises a needle adapted to in use repeatedly cycle by extending and retracting along a needle path, iv) arranging a reflective material in the path of the needle so that each time the needle extends, a piece of the reflective material is carried along the path at the tip of the needle; v) positioning the punch and/or the flexible material so that the inner surface of the flexible material is adjacent to the punch and so that in use as the needle cycles the tip of the needle at substantially full extension contacts the inner surface of the flexible material to deposit the piece of the reflective material on the inner surface of the flexible material; vi) moving either or both of the punch and flexible material, and the reflective material, so that the punch and flexible material change position relative to one another as the needle cycles and multiple reflective pieces are positioned across the inner surface of the layer of flexible material.

In an embodiment, in the step of choosing the flexible material, the material chosen is silicone- based.

In an embodiment, the method comprises the further step of heat-curing the silicone-based material once the reflective particles have been positioned. In an embodiment, in the step of arranging the flexible material, the layer is shaped so as to have a final thickness of substantially 1mm, substantially universally across the layer.

In an embodiment, in the step of extending and retracting the needle, the needle is moved along a substantially linear path.

In an embodiment, in the steps of preparing a punch apparatus and arranging a reflective material in the path of the needle the punch is chosen to further comprise a mechanism configured to receive a length of mylar tape and to spool the tape linearly through the punch in use, the needle and tape configured so that in the step of cycling the needle, the flat tip of the needle moves through the tape and presses mylar particles out from the length of tape.

In an embodiment, the needle chosen comprises a flat-tipped needle.

In an embodiment, the needle is chosen so that the glitter particles have a diameter of substantially 0.3mm.

In an embodiment, the mylar tape is chosen to be shiny.

In an embodiment, the mylar tape is chosen to have a satin or matt finish.

In an embodiment, the mylar tape is chosen to have a thickness of substantially 30um.

In an embodiment, in the steps of preparing and positioning the punch, the punch is configured so that the needle moves linearly substantially vertically.

In an embodiment, in the step of moving either or both of the punch and flexible material and the reflective material, the movement is carried out through a three-dimensional volume.

In an embodiment, in the step of moving either or both of the punch and flexible material and the reflective material, the movement is carried out so as to ensure that the particles are arrayed substantially evenly.

In an embodiment, in the step of moving either or both of the punch and flexible material and the reflective material, the movement is carried out so that the particles are positioned substantially 0.6mm from one another.

In an embodiment, in the step of moving either or both of the punch and flexible material and the reflective material, the movement is carried out so that the particles are arranged in a grid.

In an embodiment, in the step of moving either or both of the punch and flexible material, one or both of the punch or material is/are mounted on a 6-axis robot.

In an embodiment, in the step of arranging the flexible material the flexible material is mounted in a mould. In an embodiment, the mould is chosen so that the outer surface of the layer of silicone substantially has the shape of the outer/lower part of a human fingertip.

In an embodiment, the mould is mounted on the 6-axis robot.

In an embodiment, the punch is substantially stationary in use.

In an embodiment, the method of manufacturing a tactile sensor comprises the further step of adding a substantially transparent layer over the inner surface of the flexible layer.

In a second aspect the present invention may broadly be said to consist in an apparatus for manufacturing a tactile sensor, comprising: a mould configured to produce the required outermost shape for at least part of the sensor; a punch mechanism comprising a needle configured to move substantially linearly; a moving means configured to move the punch and mould relative to one another through a three-dimensional volume; the needle adapted to in use move substantially linearly towards and away from the mould so as to place a glitter particle on the inner surface of a layer of silicone in the mould each time the needle moves towards the layer, the moving means and needle operating in use so that as the mould and punch move transversely relative to one another, a plurality of glitter particles are positioned on the inner surface of the silicone layer as required

In an embodiment, the mould is configured so that the outer surface of the layer of silicone has substantially the shape of the outer/lower part of a human fingertip.

In an embodiment, the punch mechanism further comprises a tape mechanism configured to receive a length of tape, and to spool the tape linearly through the punch in use, the needle and tape configured so that in use as the needle moves linearly towards and away from the mould, the tip of the needle passes through the tape to press glitter particles out from the body of the length of tape.

In an embodiment, the apparatus further comprises a length of mylar tape, the tape passing through the tape mechanism in use.

In an embodiment, the needle is configured so that the glitter particles have a diameter of substantially 0.3mm.

In an embodiment, the needle is flat-tipped.

In an embodiment, the mylar tape is shiny.

In an embodiment, the mylar tape has a satin or matt finish.

In an embodiment, the mylar tape has a thickness of substantially 30um.

In an embodiment, the punch mechanism is stationary in use, and the moving means is configured to move the mould relative to the punch. In an embodiment, the moving means comprises a 6-axis robot.

With respect to the above description then, it is to be realised that the optimum dimensional relationships for the parts of the invention, to include variations in size, materials, shape, form, function and manner of operation, assembly and use, are deemed readily apparent and obvious to one skilled in the art, and all equivalent relationships to those illustrated in the drawings and described in the specification are intended to be encompassed by the present invention.

This invention may also be said broadly to consist in the parts, elements and features referred to or indicated in the specification of the application, individually or collectively, and any or all combinations of any two or more said parts, elements or features, and where specific integers are mentioned herein which have known equivalents in the art to which this invention relates, such known equivalents are deemed to be incorporated herein as if individually set forth.

Therefore, the foregoing is considered as illustrative only of the principles of the invention. Further, since numerous modifications and changes will readily occur to those skilled in the art, it is not desired to limit the invention to the exact construction and operation shown and described, and accordingly, all suitable modifications and equivalents may be resorted to, falling within the scope of the invention.

BRIEF DESCRIPTION OF THE FIGURES

Further aspects of the invention will become apparent from the following description which is given by way of example only and with reference to the accompanying drawings which show an embodiment of the device by way of example, and in which:

Figure 1 shows a simplified schematic view of an embodiment of the tactile sensor of the present invention, the tactile sensor comprising a flexible opaque outer ‘skin’ layer, a flexible transparent central layer, a substantially rigid transparent inner layer, a camera located behind the inner layer, and small pieces of reflective material located on the interface between the flexible opaque outer layer and the flexible transparent central layer.

Figure 2a shows a simplified perspective schematic view of an embodiment of the tactile sensor of the present invention, the sensor of this embodiment using dual cameras and having a flexible transparent central layer, a substantially rigid transparent inner layer, and a dual-layer outer skin formed from two firm opaque layers, with small pieces of reflective material located on the interface between the inner surface of the outer skin layers between the skin layers and the flexible transparent central layer, the cameras located behind and looking through the rigid transparent inner layer. Figure 2b shows a simplified schematic side cross-sectional view of the multi-camera embodiment of a variation of the tactile sensor of figure 2a in operation, with the outer surface of the outer layer making contact with an object and deforming, LED light sources positioned at the rear of the firm layer beside/between the cameras to provide a light source for the cameras and reflective pieces.

Figure 3a shows a simplified perspective schematic view of another embodiment of the tactile sensor of the present invention, the sensor of this embodiment having substantially the same features as the sensor of figure 2a, but a single-layer outer skin formed from a single silicone layer.

Figure 3b shows a simplified schematic side cross-sectional view of the multi-camera embodiment of a variation of the tactile sensor of figure 3a in operation, with the outer surface of the outer layer making contact with an object and deforming, LED light sources positioned at the rear of the firm layer beside/between the cameras to provide a light source for the cameras and reflective pieces.

Figure 4 shows a simplified schematic side cross-sectional view of an embodiment of the tactile sensor in use embedded in and forming part of the end joint of the finger of a robot hand.

Figure 5 shows top, side-rear-and-above perspective, and side-above-perspective views of the distribution of glitter on the inner surface of the flexible opaque inner layer in the embodiment of tactile sensor of figure 5, the glitter distribution generally having the same shape as the outer/lower part of a human fingertip.

Figures 6a and 6b show an embodiment of a punch used to create and position glitter pieces, the punch comprising a die and flat-topped needle, with a thin strip of mylar tape shown running through a slot in the die, the die shown in section view so as to show the position and operation of the tape and needle, the die and needles configured so that the needle can retract below the tape, and also extend upwards to reach the skin on the inside of the mould.

Figure 7 shows a front view of a punch apparatus that holds the punch in use, and which feeds tape through the punch and drives the needle, the punch apparatus comprising a body or main plate that holds the punch, motors located on the main plate to drive the punch and advance and spool the tape, a motor-driven cam rotating to lift and lower the needle, pinch rollers either side of the central punch advancing and tensioning the tape as this spools from a source spool to a destination spool.

Figure 8 shows a perspective front view of the punch apparatus of figure 7. Figure 9 shows a rear view of the punch apparatus of figures 7 and 8.

Figure 10 shows a perspective cutaway/cross-section view of a mould used to form an outer skin layer, the mould shown with outer and inner skin layers in position in the mould.

Figure 11 shows the mould of figure 10 attached to the end of a robot arm, and the punch apparatus of figures 7 to 9 next to the robot arm, the robot arm in use moving the mould relative to the punch apparatus.

Figures 12a and 12b shows a schematic representation of the paths that light from two light sources within the sensor takes, the light travelling to reflect from the glitter, back into and around the camera, figure 12a showing an idealised version to demonstrate the operating principle, where all of the light is reflected back to the camera, and figure 12b showing a more realistic version, where some reflections do not hit the centre of the camera, and some miss the camera slightly to varying degrees.

Figures 13a and 13b show an alternative embodiment of needle, having a profiled tip instead of a flat tip, the figures showing the needle in side view, the view of the needle of figure 13b showing the needle rotated 45 degrees around it’s axis as compared to the view of the needle of figure 13a.

DETAILED DESCRIPTION

Embodiments of the invention, and variations thereof, will now be described in detail with reference to the figures.

Sensor structure

The general structure of the preferred form of sensor formed by the method of the present invention comprises a thin, opaque silicone skin, containing a thick layer of clear soft flesh. A stereo pair of cameras observe the inside surface of the skin, which is covered with more than a thousand particles of glitter. The inside surface of the skin is illuminated by several LEDs. Any pressure on the outside surface of the skin produces two results:

1. The glitter moves macroscopically, since it is stuck to the skin

2. The glitter ‘sparkles’ as it moves and changes angle of alignment relative to the LEDs, even with very low forces.

The cameras are therefore able to observe minute changes in the applied force, and also calculate the shape of the skin, by stereoscopically triangulating the locations of the glitter particles.

Figure 1 shows a simplified schematic cross-sectional view of the general structure of an embodiment of the tactile sensor of the present invention, to show the principles of construction and operation. The tactile sensor 10 in this simplified schematic cross-sectional view has three main parts: a flexible opaque outer layer or skin layer 20, a flexible transparent central layer 30, a substantially rigid transparent inner layer 40, and a camera or cameras 50. Small pieces of reflective material 62, heretoforward referred to as glitter 62 or glitter pieces 62, are positioned between the skin layer 20 and the transparent central layer 30.

The camera 50 faces the rear surface of the substantially rigid transparent layer 40 (that is, the surface of layer 40 opposite to the second interface surface 70). The camera 50 captures an image of the first interface surface 60 between the flexible opaque outer skin layer 20 and the flexible transparent layer 30, including the glitter pieces 62. One or more light sources (not shown in figure 1) are provided to illuminate the first interface surface 60 of the flexible opaque layer 20 and the glitter pieces 62 through the substantially rigid transparent layer 40 and the flexible transparent inner layer 30, to allow a brightly-lit image to be captured. The image is transmitted away from the tactile sensor 10 for analysis.

In preferred forms, the flexible opaque outer layer 20 is made from an opaque elastic material (black silicone, in a preferred embodiment), and the flexible transparent layer 30 is made from a transparent elastic material. These two layers are in contact with each other at a first interface surface 60.

The opaque layer 20 is sufficiently opaque that changes in external lighting are not detectable by the camera 50. That is, during typical operations, the change in incident light on the outside surface of the sensor produces less than one bit of signal change in the cameras. For a camera that produces 8 bits per pixel, this means that the change in the amount of light entering the sensor in an area covered by one pixel from the outside is less than 1/256th of the amount of light that a pixel would see inside the sensor. That is, substantially blocking about 99% of the light is sufficient.

In the preferred embodiment, the hardness of the skin (outer layer 20) falls in the range 40- 100 Shore A.

The skin and flesh layers (flexible opaque outer layer or skin 20, flexible transparent central layer 30, substantially rigid transparent inner layer 40) are formed from an elastomer, typically silicone or polyurethane. In the preferred embodiment, silicone is used because the skin (outer layer 20) can be made from a silicone pastry - that is, a form of silicone that is plastically deformable, highly non-elastic, and non-flowing before it is cured. In this state, the silicone skin in the mould is sticky enough to accept glitter particles 62 (see below for a detailed explanation of the manufacture and placement of the glitter particles). During the cure process, the skin or outer layer 20 will bond reliably to the optically clear silicone flesh (central layer 30), and to silicone-based adhesives which are used to bond it to a metal substrate that forms part of the general structure of an end effector, robotic manipulator, or similar, of which the tactile sensor forms a part. After curing, the skin or outer layer 20 becomes elastically deformable, and robust.

The skin layer 20 is typically about 1 %-10% of the total thickness of the elastomer layers.

The substantially rigid transparent inner layer 40 may be formed from any suitable material, such as glass, but is preferably made from a plastic material, for lightness and robustness. The flexible transparent layer 30 and the substantially rigid transparent layer 40 contact each other at a second interface surface 70. In variations, the transparent rigid material (layer 40) may simply be the lens of the camera or cameras, with the surrounding rigid material being opaque.

The flexible opaque outer ‘skin’ layer 20 has a free external surface 22 on the opposite side to the first interface surface 60. This free surface 22 is the external contact surface of the tactile sensor 10, and in use will come into contact with an object to be sensed. The elasticities of the skin 20 and the flexible transparent layer 30 are such that when the external contact surface comes into contact with an object to be sensed, the first interface surface 60 between the layers 20 and 30 will be deformed as the skin layer 20 and the flexible transparent central layer 30 are pressed against the substantially rigid transparent inner layer 40.

It is most preferred that at the lower end of the range of contact force, any force just sufficient to cause just enough deformation of the skin layer 20, and thus rotation/movement of one or more glitter particles, will allow the camera or cameras 50 to detect a change in brightness from the light reflecting from the glitter particles 62. The flexibility of the material of the layers, and their dimensions, will impact on this. Ideally the highest level of force that the sensor might experience during use, including from accidental impacts, would not induce enough pressure in the elastomer layers to cause permanent change. Further ideally, this level of force would not cause a particle of glitter to impact the camera lens, and thus the sensor is still able to sense changes in force even at this very high force value.

The implication of these two requirements is that the sensor has an extremely high dynamic range; able to detect very small forces, while still being able to distinguish between two slightly different but very large forces.

The outer surface of the skin is not expected to wrinkle. The material is elastic enough that it will always maintain a smooth surface. As noted above, a large number of small pieces of reflective material - the glitter particles 62 - are located at the first interface surface 60, facing the substantially rigid transparent inner layer 40. The glitter particles 62 in different embodiments may be shiny, iridescent, highly refractive, may have a diffraction grating on the surface observed by the camera, and/or may have a holographic pattern. The glitter particles 62 are generally planar, and rest on the first interface surface 60, sandwiched between the flexible opaque outer layer 20 and the flexible transparent central layer 30. The glitter particles 62 can, in some embodiments be arranged in an ordered arrangement (such as a grid) and the method of emplacing the glitter particles 62 will be described in detail below. The pieces of glitter 62 are created so that they are as small as practically possible. The smaller the pieces are, the smaller are the details that the sensor will be able to resolve.

The lower limit on the size of the pieces is the limit at which they can be reliably handled and placed. The lower limit may also be set by the resolution of the camera. Ideally, each piece of glitter 62 takes up several pixels on the image sensor. The space between each particle of glitter 62 is preferably the equivalent of several pixels wide.

The lower useful limit is set by the skin thickness (thickness of layer 20). The thickness of the skin limits its own curvature, and therefore the limit of the size of details which can be resolved by the sensor. In general, the particles should not be very much smaller than 1/4 of the thickness of the skin.

In a preferred embodiment such as for example the embodiments of figure 4 (described in detail below), the particles are 0.3mm in diameter, spaced 0.3mm apart in a grid (each particle 0.6mm from it’s neighbours to each side and above and below, measured centre-to- centre), and the skin (layer 20) is 1mm thick.

In alternative embodiments, while the spacing of individual particles is as outlined above, the particles overall can be arranged so that at a longer distance they form patterns, such as for example spirals or zig-zags. This assists with uniquely identifying which particular part of the layer the camera is seeing deform in use.

Ideally, the camera 50 is of the type designed for machine vision applications, and has global shutters and high frame rates. A camera that can produce a colour image is not absolutely necessary (but can be used if required). However, a high bit depth per pixel is useful to be able to detect small changes in the intensity of light from the glitter particles. If a thermochromic pigment is mixed into the skin layer, then a colour camera is useful.

In preferred embodiments, the camera/image sensor or sensors 50 has/have a resolution of 640x480 capturing at 120 frames per second. The camera/image sensor/sensors 50 further have a global shutter and a monochrome image with 10 bits per pixel. The distance between the camera 50 and the skin surface is chosen so that the camera’s field of view covers the area of skin that needs to be sensitive, and so that the glitter particles 62 are generally in focus.

Although shiny glitter particles are used in most embodiments, it can also be useful to use particles that have a satin or matt finish to reduce glare. This assists with reducing flare or bloom from the reflections, that can make it difficult for the camera to ‘see’ neighbouring particles. A mix or pattern of shiny and matt/satin particles can be used as required.

The flexible opaque outer layer 20, the flexible transparent layer 30 and the substantially rigid transparent inner layer 40 are all generally laminar, with generally smooth surfaces. In figures 1, 2, and 3 the surfaces are shown as flat, but the shape of the surface can vary depending on the installation requirements of the sensor. For example, if the sensor is to be positioned in the tip of a finger of a robotic hand, the surfaces will be curved to correspond to the curvature of the fingertip. The embodiment shown in figure 4 has this arrangement. This embodiment is discussed in more detail below.

Accordingly, the interface surfaces 60 and 70, and in particular the first interface surface 60, will also be generally smooth when the flexible opaque outer layer 20 and the flexible transparent layer 30 are not deformed. As a result, the glitter particles 62 are normally arranged so that they are flat on the interface surface 60, and will reflect a certain amount of light from the light source into the camera 50. The image captured by the camera 50 (and in particular the reflections from the glitter particles 62) when the tactile sensor 10 is in this state, corresponds to a state where the contact surface of the tactile sensor 10 is not in contact with anything.

Figures 2a and 2b also show simplified schematic cross-sectional view of the general structure of an embodiment of the tactile sensor of the present invention, with further detail also present. Similar numbering is used in these figures to that used in figure 1 - e.g. flexible transparent layer 30 in figure 1 corresponds to flexible transparent layer 130 in figures 2 and 3, transparent layer 40 in figure 1 corresponds to transparent layer 140 in figures 2 and 3, etc. The embodiment in figures 2 and 3 uses dual cameras 50a and 50b rather than the single camera 50 of the simplified schematic form of figure 1.

In the embodiment of figures 2a and 2b, the layers are shown approximately to scale, but it will be appreciated that changes to the relative sizes can be made depending on how and where the sensor 110 is to be used. Further, it can be seen that rather than a single-layer flexible opaque outer layer 20 or skin 20, as used in the embodiment of figure 1, the embodiment of sensor 110 of figures 2 and 3 includes a dual-layer outer skin or ‘first layer’, formed from a firm opaque white layer 124 and an outermost firm opaque black layer 122, the layers 122 and 124 in combination forming the ‘first layer’ of skin 120. Small pieces 162 of reflective material are sandwiched between the firm opaque white layer 124 and a soft transparent layer 130 (which corresponds to the flexible transparent layer 30 of the sensor 10 of Figure 1).

The firm layers 122, 124 together form the outermost layer of the sensor 110, and act as an analogue to skin, with the firm opaque black layer 122 outermost, and acting as the contact surface of the sensor (analogous to the epidermis). Suitable materials from which the firm layers 122, 124 may be formed include silicone and polyurethane.

The firm opaque black layer 122 is formed from a hard-wearing material, to improve the life of the sensor. Any material having similar properties to human skin (or at least human epidermis) is suitable. However, in particular it is important that this layer should not be sticky - the material is chosen and/or treated to control the amount of grip or slip. The firm opaque black layer 122 also serves to block out external light, so that the inside of the sensor 110 remains optically consistent. The silicone material discussed above is suitable for this use.

The firm opaque white layer 124 also helps to block out external light, and provides a plain white background against which the small pieces 162 of reflective material can be imaged more easily. In addition, the firm opaque white layer 124 can in embodiments contain a thermochromic dye, allowing the sensor to measure the temperature of objects. Further, the sensor can be actively warmed by a heating element or heating mechanism (not shown), which would allow the sensor to detect thermal conductivity in objects in a similar manner as humans do.

The firm layers 122, 124 are relatively thin in comparison to the flexible transparent layer 130 and the substantially rigid transparent layer 140. In one particular embodiment, the firm layers 122, 124 have a combined thickness of around 1mm, the flexible transparent layer 130 has a thickness of around 5mm, and the substantially rigid transparent layer 140 has a thickness of around 3mm, but the thicknesses can of course be varied depending on circumstances.

Furthermore, the flexible transparent layer 130 is very soft relative to the firm layers 122,

124. Thus, this layer 130 corresponds to the subcutaneous layer or hypodermis, and allows the firm layers 122, 124 to move when forces are applied to the tactile surface of the sensor 110. The flexible transparent layer 130 is optically clear and colourless, so that the small pieces 162 of reflective material can be imaged. As with the firm layers 122, 124, the flexible transparent layer 130 may be formed from silicone or polyurethane. As for the embodiment of figure 1, the principal purpose of the flexible transparent layer 130 is to space the small pieces of reflective material 162 from the substantially rigid transparent layer 140; if the small pieces of reflective material 162 are in contact with the substantially rigid transparent layer 140, then they will not be able to move when an object came into contact with the sensor 110, and the sensor 110 will not function. In variations of this embodiment, a clear liquid can be used to separate the firm layers 122, 124 (and so the small pieces 162 of reflective material) from the substantially rigid transparent layer 140. In other embodiments, an air gap can be used to separate the firm layers 122, 124 and the substantially rigid transparent layer 140.

The spacing is chosen according to the needs of the sensor. In a preferred embodiment, the thickness of the soft transparent layer is approximately 15mm.

The principal purpose of the flexible transparent layer 130 is to translate forces at the interface surface into movement of the small pieces 162 of reflective material. Ideally, the flexible transparent layer 130 should be as thick as possible within the physical constraints of the sensor. A thicker layer gives more movement of the small pieces 162 of reflective material for the same amount of force applied to the outer surface.

The rigid transparent layer 140 allows the sensor 110 to be mechanically connected to whatever form of end effector it is to be used on (such as a finger of a robotic hand, as in the embodiment shown in figure 4). The layer 140 also accommodates and protect the cameras 150, light sources 190, associated electronics, and the like.

Behind the substantially rigid transparent layer 140 are located two cameras 50a and 50b and a plurality of light sources 190 (in the embodiment shown in figure 4, there are three light sources 190. For clarity, these are not shown). The regions between the cameras 50a, 50b and light sources 190 are dark, which is achieved by placing a dark coating 200 on the rear surface of the substantially rigid transparent layer 140.

The brightness of the light sources is chosen to be bright enough to be substantially brighter than any light leaking in through the opaque skin, while not overwhelming the sensitivity of the image sensors.

As shown in figure 2b, when the contact surface comes into contact with an object to be sensed, the first interface surface 160 will be deformed. During this deformation, the inclination of some of the glitter particles 162 relative to the substantially rigid transparent layer 140 will change, and as a result, the amount of light reflected back to the camera by these pieces 162 of reflective material will also change. This is shown in an exaggerated manner in the embodiment of Figure 2b, which shows schematically the change in the orientation of the reflective pieces, which will cause the change in the angle and therefore the amount of the reflected light received by the camera or cameras. Thus, the image captured by the cameras 50a, 50b will change when an object such as for example the object 80 shown in figure 2b comes into contact with the contact surface. If the reflective material pieces 162 have a holographic pattern, then the colour of light reflected will also change as the angle of inclination changes, which can assist with improving the sensitivity of the sensor.

As mentioned above, the image captured by the cameras 50a, 50b is analysed. In particular, any changes from the image captured when the first interface surface 160 is in its undeformed state (that is, when the contact surface of the tactile sensor 110 is not in contact with an object) will indicate that an object has come into contact with the contact surface of the tactile sensor 110.

The use of two or more spatially separate cameras means that stereo images of the first interface surface 160 and the glitter pieces 162 can be obtained. A 3-D image of the part of the object in contact with the contact surface can be derived from this stereo image.

The small pieces 162 of reflective material provide for tactile sensing in two principal ways. Firstly, as they act as mirrors, they are optically very sensitive to small changes in angle.

The cameras 50a, 50b can ‘see’ the light sources 190 reflected in the small pieces 162 of reflective material. A small change in the angle of a piece 162 of reflective material dramatically changes the amount of light reflected back into the cameras 150, changing from dim to bright, or vice versa, so that the pieces 162 of reflective material will ‘sparkle’ or, from any fixed position (e.g. the camera position) they will appear to produce flashes of light as their position changes. Since there are two cameras 150, and because in any given area under motion multiple pieces 162 of reflective material will be affected, there are in use multiple movements of the pieces of reflective material 162, and multiple sparkles. This feature makes the design extremely sensitive to small forces.

Secondly, as the small pieces 162 of reflective material contrast well against the white background 124, and have a clear space between them, the small pieces 162 of reflective material are easy to track in the camera images using computer vision software. Using two cameras 150, the exact position of each particle in 3D space can be triangulated. In this way, the geometry of the inside surface of the firm layers 122, 124 (the skin) can be re created inside the computer. This geometry can then be used to recreate the shape of any object pressing into the skin surface, in that it is possible to feel the difference between a flat surface, a lump, an edge, a point, or multiple points.

Further, careful design of the light sources 190 (e.g. position and brightness/type of light) can optimise the amount of sparkle, and therefore the sensitivity of the skin of the sensor 110 to small forces. A combination of several light sources 190, with dark areas 200 located between them, creates a greater amount of sparkle than a single small or large light source, and it is for this reason that the regions between the cameras 150 and light sources 190 are dark.

The design (type and position) of the light sources can affect the amount of sparkle, and thus the sensitivity of the sensor to very small forces. For all embodiments, the aim of the design of the light sources is to minimise the movement that the glitter pieces 162 need to make before that movement is detected by the image sensors. Ideally, the cameras should see one of the small light sources reflected in the glitter particle. When the particle rotates slightly due to a force being applied at the interface surface, the camera will no longer see the reflected light, and so the particle seems to suddenly dim. If the light source is larger, then the particle will have to rotate further before the light source can no longer be seen by the camera. If the light source is small, then the particle need only rotate a small way before the light source is no longer visible. Therefore, small light sources provide more sparkle, and make the sensor more sensitive to small forces.

A variation or further embodiment of the tactile sensor of the present invention is shown in figures 3a and 3b. The embodiment shown in these figures is substantially the same as that shown in figures 2a and 2b, and similar numbering is used in these figures to that used in figures 1, 2a and 2b - e.g. flexible transparent layer 30 in figure 1 corresponds to flexible transparent layer 230 in figures 3a and 3b, transparent layer 40 in figure 1 corresponds to transparent layer 240 in figures 3a and 3b, etc.

In the embodiment shown in figures 3a and 3b, the dual-layer outer skin of the embodiment of figures 2a and 2b is replaced with a single layer 220, formed from black silicone or similar.

As noted above, figures 1 to 3 show simplified schematic views, in order to best illustrate the principles behind the tactile sensor of the present invention. Figure 4 shows an embodiment of the tactile sensor in use with the finger of a robot hand. Similar numbering is used in this figure to that used in figures 1, 2, and 3 - e.g. flexible transparent layer 430 in figure 4 corresponds to flexible transparent layer 130 in figures 2 and 3, skin 420 corresponds to skin 20 in figure 1, skin layers 422, 424 correspond to firm layers 122, 124 in figures 2, and 3, the glitter pieces 462 in figure 4 correspond to glitter pieces 162, etc. Where preferred dimensions have been referred to above - e.g. the spacing of the glitter particles or the thicknesses of the layers and skin, these should be assumed to also apply to the embodiment of figure 4. A light source or sources, equivalent to sources 190, is present in this embodiment, but is not shown in figure 4. It should further be noted that the embodiment shown in figure 4 has two skin layers 422, 424 corresponding to firm layers 122, 124 as outlined above. However, in variations, the two layers 422, 424 could be replaced with a single layer, in a similar manner as the replacement of the dual-layer outer skin of the embodiment of figures 2a and 2b with a single layer 220. The single layer replacement of the two layers 422, 424 could be formed from black silicone or similar.

This embodiment of sensor 410 is bonded to a metal substrate 480 that forms a frame or skeleton for the sensor 410, the sensor 410 being roughly analogous to a fingertip on the end of a finger, and approximately the same size as a distal finger joint

The skin of the sensor 410 in this embodiment is formed from two layers, a firm opaque white layer 424 and a firm opaque outermost black layer 422, similar to the two layers of the embodiment of figures 2 and 3 described above. The firm layers 422, 424 form the outermost layer of the sensor 410, and act as an analogue to skin, with the firm opaque black layer 422 acting as the contact surface of the sensor (analogous to the epidermis). Suitable materials from which the firm layers 422, 424 may be formed include silicone and polyurethane. The combined thickness of the layers 422, 424 is approximately 1mm.

The firm opaque black layer 422 is formed from a hard-wearing material, to improve the life of the sensor. Any material having similar properties to human skin (or at least human epidermis) is suitable. However, in particular it is important that this layer should not be sticky - the material is chosen and/or treated to control the amount of grip or slip. The firm opaque black layer 422 also serves to block out external light, so that the inside of the sensor 410 remains optically consistent. The silicone material discussed above is suitable for this use.

A soft transparent layer 430 is located behind the opaque white layer 424. The layer 430 fills most of the space in the ‘fingertip’, between the inner surface of the layer 424, and the metal substrate 480. As for the previous embodiments, the layer 430 is very soft relative to the firm layers 422 and 424.

A layer of glitter pieces 462 is sandwiched between the firm opaque white layer 424 and a soft transparent layer 430. The pieces are spread over the inner surface of layer 424 substantially evenly. As shown in figure 5, the general shape of these surfaces is a curved ‘scoop’ that generally has the shape of the outer/lower part of a human fingertip.

An image sensor assembly or camera 450 is located at approximately the equivalent position as a fingernail would be in a human hand. The sensor has a lens 451 at the ‘base’ of the fingernail, that is aligned to point outwards and forward diagonally from the ‘base’, towards the skin formed by layers 422 and 424, these layers curving around the field of view of the lens 451 as shown in figure 5. In the preferred embodiments, the camera lens or lenses 451 is/are formed from sapphire. Sapphire has a higher refractive index than glass, and therefore light passing from the soft transparent silicone layer into the lens will bend (the silicone layer has a similar refractive index to glass, and therefore light passing from the silicone layer into a glass lens will not bend, and the other surface of the lens will have no effect on the light path). An image sensor unit 470 is located at the rear of the sensor assembly/camera 450, hard-connected to a remotely-located image processor (not shown). Ideally, the camera 450 is of the type designed for machine vision applications, and has global shutter and high frame rate. A colour image is not necessary in this application, but a high bit depth per pixel is useful to be able to detect small changes in the intensity of light from the glitter particles. If a thermochromic pigment is mixed into the skin layer, then a colour camera is useful. In this embodiment, the camera/image sensor 450 has a resolution of 640x480 capturing at 120 frames per second. The camera/image sensor 450 further has a global shutter and a monochrome image with 10 bits per pixel. The distance between the camera and the skin surface is chosen so that the camera’s field of view covers the area of skin that needs to be sensitive, and so that the glitter particles are generally in focus.

The operation of the sensor 410 is the same or similar to the embodiments described above: The camera 450 captures an image of the rear surface of layer 424, on which the glitter 462 is distributed. One or more light sources (not shown) are provided to illuminate this surface and the small reflective pieces of glitter 462, to allow a brightly-lit image to be captured. The image is captured by the image sensor unit 470 and transmitted for analysis. This initial image corresponds to a neutral position, where nothing is being touched or held.

As mentioned above, when the contact surface - that is, the outer surface of layer 422 - comes into contact with an object to be sensed, the layers 424 and 422 are deformed away from their neutral position. During this deformation, the inclination of some of the small pieces of reflective material 462 will change, and as a result, the amount of light reflected back to the camera 450 by these pieces will also change, and the image captured by the camera 450 will change. The new (deformation) image captured by the camera 450 is analysed. In particular, any changes between the ‘neutral’ or ‘undeformed’ image and the ‘deformed’ image. Analysis of these changes shows the shape of the object and how much ‘grip’ force is being applied, as outlined in the analysis section below. The paths of the light are shown schematically in figures 12a and 12b.

As shown in these figures, the light from two light sources 490 within the sensor passes from the light sources to the glitter, where it is reflected back into and around the camera as the glitter moves, as the layers are deformed by pressing onto an external source.

Figure 12a shows an idealised version where all of the light is reflected back to the camera, so as to illustrate the operating principle. Figure 12b shows a version that is closer to the actual operating conditions, where some reflections from the glitter pieces do not hit the centre of the camera, and some miss the camera slightly to varying degrees. The light from these ‘misses’ and ‘off-centre’ pieces will potentially still be detected by the camera, depending on how far the reflection is bent away from the camera. However, the intensity will be less than a direct reflection.

In operation, some light can be reflected back from layer 424. This reflected light can then be re-reflected back by the substrate 480 to further illuminate the white inside layer of skin 424. This is often an undesirable result, and can be minimised or eliminated by colouring the substrate layer 480 black, so that very little extra light re-reflects onto the inside of the skin.

Glitter manufacture and placement

The manner in which the glitter particles are formed and positioned will now be described, with reference to figures 5 to 11.

As noted above, for the embodiment of figure 4, a layer of glitter pieces 462 is sandwiched between the firm opaque white layer 424 and a soft transparent layer 430. The pieces are spread over the inner surface of layer 424, spaced 0.3mm apart in a grid (each particle 0.6mm from it’s neighbours to each side and above and below, measured centre-to-centre), and the skin (layer 20) is 1mm thick.

As shown in figure 5, the general shape of these surfaces is a curved ‘scoop’ that generally has the shape of the outer/lower part of a human fingertip.

The glitter particles 462 are very preferably substantially the same size and shape as each other. It is also highly preferred that the particles 462 are arrayed substantially evenly and neatly on the inside surface of the skin layer 424 - that is, spaced 0.3mm apart in a grid (each particle 0.6mm from it’s neighbours to each side and above and below). However, any ordered pattern or even random or semi-random placement could be achieved as required.

In order to achieve this, a punch mechanism is used which is able to punch 0.3mm diameter discs of 30um thick mylar. The punched discs (glitter pieces 462) are then placed on the uncured silicone skin that forms layer 424.

An embodiment of a punch 500 used to create the glitter pieces 462 is shown in figures 6a and 6b. The punch 500 comprises a die 501 and flat-topped needle 502. In use, a thin strip of shiny mylar tape 503 runs through a slot in the die. The die 501 is shown in section view in figures 6a and 6b, to show the position and operation of the tape 503 and needle 502. The needle 502 can retract into the die 501 below the tape, and also extend upwards through the tape to reach the skin on the inside of the mould - that is, in use the needle 502 cycles over a substantially linear path, moving from one end (retracted) of the linear path to another (extended upwards), in a movement cycle - that is, repeated movements along the path. As used in this specification, ‘cycle’ should not be taken as meaning that each cycle has to be the same as all others - each cycle is generally the same as the others, but the length of the cycles could vary from one another within an overall operation that comprises multiple cycles. In use, as the needle 501 extends and retracts, the needle tip punches through the tape 503 to punch a 0.3mm diameter disc out of the strip of tape 503. The punched disc is a single particle of the glitter - a glitter piece 462 - which sits on top of the tip of the needle 502. In variations, the tip of the needle could be profiled or shaped to punch pieces out of the tape of the preferred size and shape for a particular application. An example of an alternative embodiment of profiled needle is shown in figure 13.

The punch 500 is used as part of a larger punch apparatus 510, shown in figures 7, 8, and 9.

As shown in figure 7, the punch apparatus 510 comprises a body or main plate 511 to hold the punch 500. Motors 512 are located on the main plate 511 to drive the punch 500 and advance and spool the tape 503. A motor-driven cam 513 rotates, to lift and lower the needle 502 of the punch 500 as the cam 513 rotates. Pinch rollers 514 either side of the central punch advance and tension the tape 503, which spools from a source spool 515 to a destination spool 516. A reciprocating tape guide 517 guides the tape onto the destination spool 516.

All actions (spooling, punching, tape advancing, and guiding) are actuated by the stepper motors 512, located on the opposite side of the main plate 511 to the punch 500, cam 513, pinch rollers 514, rollers 514, spools 515 and 516, and tape guide 517. The motors 512 are in turn controlled by a controller board 518 mounted on the plate 511 next to the motors 512.

As outlined above, the skin of the sensor comprises a layer of silicone pastry, which is non flowing but deformable before curing, and which can be pressed into a mould, then heat cured. After curing, the silicone skin is a durable rubber.

The skin 420 in the preferred embodiment as shown in figure 4 is made up of two layers of silicone - the black outer layer 422 and white inner layer 424.

As shown in figure 10, the skin is formed by laying this into a mould 600, with a mould core (not shown) pressed in on top of the skin, so as to push and force the skin into the right shape and thickness. The core is then removed, leaving the skin in the mould. Figure 10 shows the mould 600 in cross-section so that the outer and inner surfaces 422, 424 of the skin can be seen. In use, and as shown in figure 11, the mould 600 is mounted on a 6-axis robot arm 601, which offers the mould up to the punch 500, with the skin layers positioned within it. As the punch 500 operates (as described above), with the needle 502 moving upwards and downwards substantially vertically, the robot arm moves the mould 600 relative to the punch 500, so that individual glitter particles 462 are placed onto the inner surface of the silicone skin - the inner surface of layer 424 - as they are punched out of the ribbon/tape 503. As noted above, the silicone skin in the mould is sticky enough to accept glitter particles 462 before it is fully cured, and the glitter particles 462 stick to the surface and are held permanently in position as the skin cures. In variations, any 6-axis robot could be used.

The method above specifically describes how to create a ‘fingertip’ sensor the same as or similar to that shown in figure 4. It should be noted that the steps of the method can also be more generally applied, to create variations or different embodiments of sensor.

That is, the method in a general form is to choose a suitable flexible surface material (such as the skin 420 above); and to apply small reflective pieces (e.g. glitter pieces 462) to this flexible surface. The completed material with the applied reflective pieces can then be incorporated into a sensing system as part of the system, so that the reflective pieces will reflect light when the flexible material is deformed, for example when contacting an external object or otherwise having force applied to the outer surface.

The preferred manner in which the reflective pieces are applied is as outlined above, via the use of a needle or similar punching apparatus passing through a larger sheet or ribbon to create small reflective pieces, and to position these on the surface of a continuous layer, sheet or stratum of the flexible material. As outlined above, the sheet or ribbon moves and the punch remains stationary. However, in variations, the sheet or ribbon could remain stationary, with the needle moving relative to this. As outlined above, the sheet in the preferred embodiment is of substantially continuous and uniform thickness over it’s area, but could include protrusions or troughs, or be formed into a wav-like pattern or similar. ‘Sheet’ or ‘layer’ as used in this specification should not be taken as meaning planar or flat. So for example layer 420 has the general shape of a curved ‘scoop’, as described above.

The embodiments of tactile sensor described above are described in the context of use on a robotic end effector; however, it will be appreciated that the tactile sensor is not limited to this, and can be used in other situations, such as for example Game controllers, Virtual reality interfaces, other parts of the robot.