Field of the invention

The present invention relates to a handheld guidance device for the visually-impaired.

Background to the invention

Visually-impaired or blind people can employ a "white cane" as a means of navigating their environment, locating objects, and avoiding obstacles. Whilst the white cane is a widespread and simple aid, that can provide useful tactile feedback about a user's environment, it has a number of drawbacks.

For example, when used as a walking aid, the white cane can deliver tactile feedback about the environment immediately in front of and below the user, but cannot provide a warning about raised obstacles such as low hanging branches. It is also difficult to determine information about the height or depth of obstacles.

It is against this background that the present invention has been devised.

Summary of the invention

According to a first aspect of the present invention there may be provided a device for guiding the visually-impaired. The device may be a handheld guidance device. The device may comprise at least one of: o a connector for connecting the guidance device to a cane; o a sensor array for capturing images of an environment forward of the user; o a processor configured to process the captured images to determine guidance information about objects within that environment; and o a non-visual user interface configured to feedback guidance information to a user.

Preferably, the guidance information includes at least one of: o a description of an object; o a position of an object relative to the user; and o a distance of an object from the user.

Preferably, the sensor array is positioned and arranged relative to the connector, in normal use of the device and/or cane, to capture images of the environment forward of the user. The device may further comprise a memory. A library of models of predetermined objects may be stored in the memory. The processor may be configured to process the captured images in accordance with an image recognition routine. The image recognition routine may involve capturing images via the sensor array, and then comparing them with the library of models to determine a match with an object within the library, and thereby to determine guidance information containing a description of a matched object, such as the name of the object. The library of models may be generated, at least in part, by the supervised training of a neural network.

The device may further comprises at least one of an inertial measurement unit (IMU) and a positioning module. The IMU and/or the positioning module may be in communication with the processor, the processor being configured to process the captured images to determine guidance information in dependence on the position, orientation and/or speed of device as detected by the IMU and/or positioning module.

The device may be configured to download a temporary addition to the library of models in dependence on the location of the device. To this end, the device may comprise a wireless telecommunication module configured to wirelessly download the library of models, or additions thereto. The temporary addition to the library of models may include models of objects in the area surrounding that location, such as shop logos, and landmarks.

Preferably, the processor is configured to process the captured images to determine guidance information about objects within that environment, the processing being dependent on the position and/or size of those objects within a field of view of the sensor array.

Preferably, the sensor array comprises a depth sensor for determining distances to objects. Preferably, the sensor array comprises one or more visible light camera. The depth sensor and the one or more visible light camera may have overlapping fields of view. Where there are multiple visible light cameras, it is preferred that they each have a different field of view, with only minor overlaps between them.

Preferably, the device is generally elongate along an axis. Ideally, the connector is arranged to hold a cane along an axis substantially parallel to that of the device. The connector may be arranged for hand-detachable attachment of the device to a cane.

Preferably, the connector comprises a clamping portion arranged for clamping around the cane. Preferably, the connector comprises a mounting plate affixed to a casing of the device. The clamping portion and the mounting plate may be hand-detachably attachable to one another via complementary engagement formations. The complementary engagement formations may define a sliding joint.

Preferably, the user interface comprises a plurality of tactile feedback nodules actuated under control of the processor to provide a tactile feedback pattern dependent on at least one of the position and distance of an object relative to the user. Preferably, the tactile feedback nodules are controlled to change their tactile feedback characteristics proportionally to the proximity of an obstacle. For example, a frequency of vibration may increase with closer proximity.

The tactile feedback nodules may be actuated under control of the processor to provide a tactile feedback pattern dependent on whether an object is determined by the processor to belong to a predetermined type of object. A predetermined type of object may be an obstacle and/or a crossing. Thus, the processor may be configured to determine both what the object is, and what its classification or type is. Accordingly, the processor may be configured to execute a classification routine for classifying objects detected from captured images into one of a plurality of predetermined types.

The tactile feedback nodules may be located on a handle of the device. The handle may be contoured to define finger-receiving portions, each accommodating a respective tactile feedback nodule.

Preferably, each tactile feedback nodule comprises a vibration motor operatively connected to the processor via a circuit board. The circuit board may comprise a plurality of flexible arms, on the distal ends of which a corresponding one of the vibration motors is mounted.

Preferably, the user interface further comprises an audio transceiver for audio communication between the device and the user. An output of the audio transceiver may be configured to relay at least part of the guidance information to the user. The audio transceiver may comprise a voice input via which verbal commands can be issued from the user to the device. The input of verbal commands may be controlled via an activation mechanism, such as a specific wake-word. The device may further comprise a short- range communication module via which audio signals between the device and user can pass wirelessly via a local auxiliary device.

Preferably, a memory of the device comprises an application that, when executed, controls the operation of the processor, the device comprising a wireless telecommunication module operable to communicate periodically with a remote application server to download from it updates to the application. According to a second aspect of the present invention there may be provided a user guidance system. The user guidance system may comprise the device according to the first aspect of the present invention. The user guidance system may comprise at least one of an auxiliary local device, and a remote device, such as a server. Preferably, the user guidance device comprises a telecommunication module for communicating with said local and/or remote devices. The telecommunication module may be a wireless telecommunication module. Preferably, the auxiliary local device comprises an audio transducer, such as a bone conduction transducer, via which audio is communicated to a user.

According to a third aspect of the present invention there is provided a computer- implemented user guidance method for the visually-impaired. The method may comprises at least one of: o capturing images of an environment forward of the user; o processing captured images to determine guidance information about objects within that environment, including at least one of: o a description of an object; o a position of an object relative to the user; and o a distance of an object from the user; and o providing non-visual feedback to a user including said guidance information.

According to a fourth aspect of the present invention there is provided a computer- implemented device or system configured to implement the method according to the method of the third aspect.

It will be understood that features and advantages of different aspects of the present invention may be combined or substituted with one another where context allows.

For example, the features of the device described in relation to the first aspect of the present invention may be provided as part of the system of the second aspect, and/or the method described in relation to the third aspect of the present invention.

Furthermore, such features may themselves constitute further aspects of the present invention, either alone or in combination with others. For example, the features of the processor, the application controlling the processor, the connector, the user interface, the library of models, etc may themselves constitute further aspects of the present invention. Brief of the

In order for the invention to be more readily understood, embodiments of the invention will now be described, by way of example only, with reference to the accompanying drawings in which:

Figure 1 shows a perspective view of a handheld guidance device according to an embodiment of the present invention, in combination with a cane;

Figure 2 shows an enlarged perspective view of the device of Figure 1 in isolation;

Figure 3 shows a perspective view of component parts of a connector of the device of Figure 1 ;

Figure 4 shows an underside view of the guidance device of Figure 2 without the connector of Figure 3;

Figure 5 shows an underside view of the guidance device of Figure 2;

Figure 6 shows a front view of the guidance device of Figure 2;

Figure 7 shows a rear view of the guidance device of Figure 2;

Figure 8 shows an overhead view of the guidance device of Figure 2;

Figure 9 shows a side view of the guidance device of Figure 2;

Figure 10 is a perspective cut-away view of the guidance device of Figure 3, with part of the casing at the left side of the device being omitted to show components within the guidance device;

Figure 11 is a side cut-away view of the guidance device of Figure 10;

Figure 12 is a rear cut-away view of the guidance device of Figure 10;

Figure 13 is a front cut-away view of the guidance device of Figure 10;

Figure 14 is an underside cut-away view of the guidance device of Figure 10;

Figure 15 is an overhead cut-away view of the guidance device of Figure 10; and

Figure 16 shows a schematic block-diagram of a guidance device according of various embodiments of the present invention. Specific description of the preferred embodiments

Figure 1 shows a perspective view of a handheld guidance device 1 according to an embodiment of the present invention. The guidance device 1 is shown in combination with a cane 9, shown in dotted outline, typical of a white cane used by the visually impaired. The cane 9 has a rounded tip 9a at an end distal to a user, to allow it to be dragged across the ground in use without snagging, and the other end, proximal to a user, defines a broadly cylindrical grip 9b. White canes generally customised, at least having different lengths tailored to the height and reach of each user.

The guidance device comprises a connector 2 that defines a bore through which the broadly cylindrical proximal grip 9b of the cane 9 is retained in use. When so retained, the elongate shape of the guidance device 1 generally extends along an axis parallel to the cane 9. It should be noted that the guidance device 1 can be used without the cane 9, but there are various advantages associated with the combination.

Figure 2 shows an enlarged perspective view of the handheld guidance device 1 in isolation. The device 1 comprises a head 10 that defines the front of the device, and an elongate handle 20 that extends from the head 10 towards the rear of the device 1. The head 10 is larger in cross-section than the handle 20, defining a bulbous hammer-head- like structure within which various sensors are housed.

The connector 2 is positioned directly beneath the head 10 so that when the cane 9 is secured by the connector 2, the end of the cylindrical grip 9b can be maintained substantially forward of the handle 20 of the device 1. Thus, the cane 9 does not interfere with the ability of the user to hold the device 1 by its handle 20. At the same time the head 10, and the sensor within it, can be reliably aligned with the cane 9, and so be maintained, in use, at a relatively predictable orientation relative to the ground. Naturally, a user can manipulate the cane 9 via the handle 20 of the device 1.

Figure 3 shows a perspective view of component parts of the connector 2, including a mounting plate 3 and screws 4 for affixing the mounting plate to the underside of the head 10.

Figure 4 shows an underside view of the guidance device 1 without the connector 2. As shown, screw holes 4a are defined in the region of the head 10 to receive the screws 4 for the mounting plate 3.

Referring back to Figure 3, the connector 2, further comprises a clamping portion 5. The clamping portion 5 includes a hinge 6 about which a pair of jaws 7 are able to pivot between an open position, as shown in Figure 3, and a closed position. The open position allows insertion and removal of the cane 9, and the closed position clamps around the cane 9, defining the bore which accommodates the cane 9. To aid secure clamping, the jaws 7 define a gripping surface 8 on the part of each jaws 7 that define the inwardly- facing surface of the bore of the connector 2.

The clamping portion 5 and the mounting plate 3 define complementary engagement formations that allow a user to securely engage the clamping portion 5 to the mounting plate 3, or release it without recourse to tools. Thus, the cane 9 can be attached and detached from the guidance device 1 repeatedly, if necessary, by a user by hand.

The complementary engagement formations establish a sliding dovetail joint, with a pair of grooves 5a being disposed on the clamping portion 5, which act as a socket for a corresponding pair of flared rails 3a of the mounting plate 3 which are oriented parallel to one another. Each groove 5a is positioned on a respective jaw 7 of the clamping portion 5, so that when the jaws 7 are closed, the grooves 5a are arranged relative to one another to match the grooves 7a, and so are able to slide axially into engagement with them. The grooves 7a and/or the rails 3a are slightly tapered so that as they are pushed into engagement with one another, the frictional join between them progressively tightens.

Figures 5 to 9 shows various views of the guidance device 1 in isolation with the connector 2 attached.

Figure 5 shows an underside view of the guidance device 1. Here, in the region of the handle 20, the device 1 comprises a finger-grip region 21 which supports a resilient easy- to-grip material, such as rubber.

Likewise, referring to Figures 8 and 9, which show an overhead and side view of the guidance device 1 respectively, the top of the handle 20 also supports a palm-and-thumb- grip region 21a, also supporting the same resilient material, and so making the handle 20 of the device 1 easier to grip.

Referring to Figure 9, the underside of the handle in the finger-grip region is sinusoidally contoured to define four valleys, each for receiving a finger of a user. Within each valley, a raised tactile feedback nodule 23 is provided, one for each finger.

Two thumb buttons 22 are provided towards the front of the palm-and-thumb-grip region, positioned so that when the handle 20 of the device 1 is gripped by a user, their thumb points forward and over the buttons. Like the nodules 23, the buttons 22 are slightly raised from their underlying surface, and so more easily found and operated by touch. The device 1 has casing T that is divided into an upper half 1 U and a lower half 1 L, with each half being constructed from an integral piece of moulded plastics material.

Referring to Figure 7, which shows a rear view of the device 1 , the rear end of the handle 20, and so the casing T, is interrupted by a slot 24a for a charging port 44, and a wriststrap eyelet 24b. The latter forms an anchor point for a wrist-strap (not shown) that can be worn around the wrist of a user, providing a way of maintaining the connection between the user and the guidance device 1 in the event that the device 1 slips out of the hand of the user during use.

Referring to Figure 6, which shows a front view of the device 1 , the front, upper part of the head 10, and thus the upper part 1U of the casing T, is interrupted by a laterally- extending sensor window 25. The window 25 is translucent to the relevant sensors, as will be described, but forms a physical barrier, as the casing T does, to the interior of the device 1 , protecting the electronic components within the casing of the device 1 from ingress of contaminants and water.

Figures 10 to 15 are various cut-way views of the guidance device 1 without the connector 2. In these Figures, the part of the casing T at the left side of the device 1 is omitted to show the components inside the guidance device 1. In order, a perspective view, a leftside view, a rear view, a front view, an underside view, and an overhead view are shown in Figures 10-15.

The guidance device 1 further comprises a primary circuit board 40, a cylindrical battery 42, a charging port 44, vibration generators 46, and a sensor module 48.

The battery 42 is primarily located within the lower part of the device 1, extending almost the entire axial length of the device 1 from the front of the head 10 to the rear of the handle 20. At these endpoints at the head 10 and handle 20, the lower half 1L of the casing T is internally moulded to define recesses to retain the battery 42 in position. The battery 42 is held centrally. Accordingly, the weight of the battery 42, which is a major contributor to the weight of the device 1 as a whole, is distributed evenly. This improves the balance and handling of the device 1. In normal use, the battery, when fully-charged, can provide the guidance device 1 with a power source for 12 hours.

The primary circuit board 40 also extends axially within the device, between the sensor module 48 at the head 10, to the charging port 44 at the rear. The primary circuit board 40 is fixed into place, during assembly of the device 1, to the upper half 1U of the casing T via screws 1S that are tightened into screw-risers 1R integrally-moulded into the upper half 1U of the casing. This keeps the circuit board separated from the adjacent part of the casing T, thereby defining a cavity within which can be accommodated various electronic components mounted on the upper surface of the circuit board (not shown). The circuit board 40 defines a tail 40T, at the end of which the charging port 44 is connected, and four arms 40A, the end of each of which support one of the four corresponding vibration generators 46.

The substrate of the circuit board is constructed, at least in the region of the arms 40A, from a flexible material. The four arms 40A of the circuit board 40 can therefore, during assembly, be flexed away from the main plane of the primary circuit board 40, to loop around the battery 42, with the end of each arm 40A terminating at a position located directly adjacent a corresponding tactile feedback nodule. Each vibration generator 46 forms part of a respective tactile feedback nodule 23.

The vibration generators 46 are in the form of piezoelectric motors, that are controllable to generate vibration patterns of varying duration, intensity and frequency. Thus, through vibratory transmission via the raised part of an adjacent tactile feedback nodule 23, each can provide a rich source of tactile feedback for a user.

The sensor module 48 is positioned within the head 10 of the device, and angled relative to the plane of the primary circuit board 40 so that the sensors that it supports, as will be described in further detail below, face out of the sensor window 25. This angle is approximately 32 degrees, but in various embodiment, can ideally be between 25 and 40 degrees. Thus, in normal use of the guidance device 1 whilst the device 1 is held tilted down and in front of a user - as a white cane would be - the sensor window 25 faces directly forward, in the typical direction of travel of the user.

This leads to one of the advantages associated with combining the guidance device 1 with a cane 9. Doing so ensures that the guidance device 1 is maintained at a regular and relatively predictable position and orientation relative to both the user and the ground over which the user is walking.

Resilient clips 49 either side of the sensor module 48 hold the sensor module 48 in position by clipping into cooperating structures moulded as part of the internal surface of the upper casing half 1U.

Figure 16 shows a schematic block-diagram of the guidance device 1 of various embodiments of the invention, including those shown in the previous Figures. The device 1 further comprises an inertial measurement unit (IMU) 11, a processor 12, a memory 13, a positioning module 14, a wireless radio telecommunication module 18, and a sensor array 17. As shown schematically via dotted lines, these and other electronic components of the guidance device 1 are generally functionally interconnected for power and communication via the primary circuit board of the device 1. In particular, the processor 12 is arranged to receive information from sensors 17, 11 , 14 of the device, and output instructions and responses to help the user of the guidance device 1 determine properties of their environment, and so more safely navigate through it. For example, as already mentioned, outputs can include activation of the vibration generators 46.

The wireless telecommunication module 18 comprises a short-range communication unit, such as a Bluetooth™ transceiver, for connecting with one or more auxiliary local devices 18a, and a long-range communication unit, such as a cellular transceiver, for connecting via a network 18b such as the Internet to remote devices 18c.

The positioning module 14 is in the form of a radio-localisation module, such as a GPS module, that allows the guidance device 1 to determine its absolute location on Earth. The IMU 11 comprises a compass and a 3-axis accelerometer, allowing the heading and orientation of the guidance device to be determined by the processor 12.

The sensor module includes the sensor array 17 of cooperating sensors, and comprising an overhead camera 17a, a forward camera 17b, a ground camera 17c, and an infrared depth sensor 17d. These are positioned with their combined field of view extending approximately over 180 degrees, outward of the sensor window. Thus, objects within the forward path of a user of the guidance device can be reliably detected by the sensor array 17, including those at ground level, and those at or above head height. The sensors of the sensor array 17 are cooperative as they have at least partially overlapping fields of view. In alternative embodiments, the function provided by the overhead, forward and/or ground camera may be combined into a single camera, having a wide-angle field of view.

The depth sensor 17d comprises an infrared projector and IR camera that utilises the infrared projection patterns to determine the distance between the guidance device 1 and objects within the field of view of the IR camera. The field of view of the IR camera of the depth sensor 17d overlaps with all three other cameras 17a, 17b, 17c, the latter allowing a standard "visible light" image of an object to be captured, and the former determining a distance to that object.

In alternative embodiments, the visible light cameras 17a, 17b, 17c, and the depth sensor 17d, may be integrated in a single sensor package, such as an integrated multi-spectral optical sensor.

The memory 13 stores an application executed by the processor to govern the operation of the guidance device 1 in use as will be described. Instructions for operating the device 1 and subsequent guidance issued by the device 1 to a user for the purposes of safe navigation are stored in the memory 13 of the device 1 as audio. This audio is in the preferred language of the user.

These are played back to the user via an audio feedback transducer. Other information, such as a low-battery, may also be communicated via the audio feedback transducer.

In certain embodiments of the invention, the audio feedback transducer is internal to the guidance device, such as an internal speaker. However in the present embodiment, the audio feedback transducer is part of the auxiliary local device 18a with which the device communicates via the Bluetooth™ transceiver of the wireless telecommunication module 18. In particular, the auxiliary local device 18a comprises bone conduction audio transducers worn by the user via which audio instructions and guidance can be provided to the user. Advantageously, bone conduction audio transducers allow the user to receive clear audio feedback in a potentially noisy environment, but simultaneously do not block the sounds of that environment. The latter may be critical for any visually-impaired person to safely navigate that environment. Such sound blocking is a problem with existing on- ear or in-ear headphones.

The auxiliary local device 18a pairs with the guidance device 1 via Bluetooth™ for secure and reliable audio transmission. The auxiliary local device 18a also comprises a microphone via which the user can input verbal commands and responses to the guidance device 1 via the wireless connection. To reduce the burden of voice-processing, and increase the reliability and usability of the guidance device, the input of verbal commands via the microphone is controlled via an activation mechanism, such as a specific "wake-word".

More than one auxiliary local device 18a may be paired with the guidance device. For example, a smartphone may also be paired, and used as a conduit through which information can pass between the user and the guidance device 1 , for example using a preferred user interface that the user may have already set up on the smartphone.

In this way, a convenient and effective interface is established between the user and the guidance device 1, via which the guidance device can be set up and used by the user.

Set up may involve the user providing the guidance device 1 with personal preference or calibration information. For example, a user may enter their height. Such information assists the device 1 in determining distances such as the likely height of the guidance device 1 from the ground, and the stride length of the user. The latter can be used during audio guidance, for example, informing the user of the distance to objects in the form of the number of steps or strides to reach that object.

In any case, the guidance device 1 is able to calculate its position relative to the user, the ground and objects within the environment forward of the device, and so the position of the user holding it in use relative to those objects. Additionally, the guidance device 1 is able to determine its speed and direction of travel through a three-dimensional environment, and so is able to estimate its likely distance and position from objects ahead of time, assuming that such speed and direction of travel is maintained.

The device 1 implements an image recognition system for use in enhancing the guidance transmitted to the user. Images captured by the cameras 17a, 17b, 17c are processed by the processor 12 in accordance with an image recognition routine governed by the application stored in the memory 13. The application contains a library of models of predetermined objects, likely to be encountered within an environment. Each model comprises metadata about the object, such as the name of the object, or another description to be communicated to the user.

The routine includes comparing the images captured by the cameras 17a, 17b, 17c of the sensor array 17 with models to determine a match. In response to determining a match, the guidance device 1 provides feedback to the user about the recognised object in the environment, for example by providing an audio description of that object based on the metadata. Additionally, the calculated distance and position of that object are relative to the user may also be provided as feedback.

Object models are ideally classified by their likely location within the field of view of the sensor array. For example, objects that are likely to be found along or near the ground, near the lower part of the combined field of view of the sensor array 17, are classified under a ground classification. Similarly, forward classifications and overhead classifications are used for the other respective parts of the field of view of the sensor array 17. These classifications are utilised during a match operation to weight the probability of a match in dependence on the location of objects within the field of view of the sensor array.

In the present embodiment, the models and/or library as a whole is generated by applying machine learning technique, and in particular via the application of supervised learning. Specifically, a set of training data that includes images of already-known objects are used to supervise the training of a neural network. A sufficient size of training data set allows the general recognition of those objects from images captured by the cameras of the sensor array 17. In alternatives, different machine learning techniques can be applied to facilitate the recognition of objects within the field of view of the sensor array 17.

The appearance of an object within a captured image can also be used to estimate distance to that object. For example, where the object has a predetermined regular size, (e.g. a generic street sign) the distance from it can be inferred by the size that it occupies within the image captured by a camera of the sensor array 17. Naturally, this information can be supplemented by the depth sensor 17d, if it is within range, but the depth sensor 17d generally has a shorter range than the visible light cameras 17a, 17b, 17c.

The image recognition routine is further assisted by registering which camera has detected a given image (or otherwise, what part of the total field of view includes an image of an object recognisable via the image recognition routine). Images captured of the ground - e.g. by the ground camera 17c - have a higher probability of matching to objects within the library classified as images likely to be found along the ground, such as road markings, kerbs, tactile flooring, grates, paths, grass and low-lying vegetation.

Additionally, objects within the view of multiple cameras can be compared for their likely correlation, and this also can be used to improve the reliability of image recognition. For example, the base, the trunk, and the branches and leaves of a tree may each be detected by a respective camera of the sensor array 17. Comparing the occurrence of those objects along with their heading within the field of view of each camera can be used to enhance the reliable detection of the tree.

The image recognition routine also involves utilising data from the other components to supplement and improve the reliability of image recognition.

For example, data from the IMU 11 can be used to determine the precise orientation of the device and therefore the likely location of an object captured by one of the cameras. If the device is being oriented as intended then the ground camera 17c is likely to capture images of objects along the ground. However, if the device is raised with the head 10 and handle 20 approximately level, as shown in Figure 9, then the ground camera is likely to capture above-ground images normally captured by the forward camera 17b.

The positioning module 14 can also be used to enhance image recognition. The absolute location of the device 1 on Earth, as detected by the positioning module 14 can be used to determine the position of the guidance device 1 within a corresponding map of the environment. The map may be populated with the occurrence, at that location, of one of the predetermined objects that the image recognition system is capable of identifying. This information can be used to assist the image recognition routine in more reliably identifying such objects and moreover determining an accurate relative positioning of that object and the device 1. As mentioned, this information can be imparted to the user, ideally via audio feedback.

Furthermore, a specific advantageous method of operation comprises the detection, via the positioning module 14, of the location of the user device and downloading, via the wireless telecommunication module 18, a set of location-specific models for the area proximal to that detected location. These can be obtained via the network 18b from a remote server 18c configured to receive, from the device, a request for location-specific models, the request including at least the location of the guidance device 1.

The set of location-specific models are stored in the library for use in image recognition, and relate to objects that are specific, or even unique to that location. For example, the set of location-specific models may include landmarks, points of interest, and buildings such as shops. The latter, in particular, can be reliably identified via the image recognition of logos and words corresponding to the brand of the shop, as the visual appearance of such images are carefully controlled and displayed prominently, so as to promote awareness and recognition of the brand.

In a similar way, geotagged images on which image recognition has already been performed can also be used for assisting the image recognition routine in identifying objects within the user's environment. Such geotagged images may be captured by the device 1 itself and stored for later use. This can be particularly advantageous for increasing the reliability of the device for routes that are followed repeatedly by a user.

The application stored in the memory 13 of the device 1 can be updated periodically to enhance the capabilities of the image recognition routine and other functionality. The processor 12 is configured by the application to periodically contact an application server via the network 18b to determine if updates are available, and if so download those updates. Naturally this is done via the long-range module of the wireless telecommunication module 18.

Image recognition performed locally is preferred in the present embodiment as it allows recognition to be performed near-instantaneously without the necessity of a wireless signal. However, in alternative embodiments, image recognition may be performed on a server 18c remote from the location of the guidance device 1. The storage capabilities of the memory 13 of the device 1 are restricted, and so only a limited number of objects can be stored within the library used for image recognition. Thus, it is not possible to necessarily recognise every object. To address this issue, images captured by the cameras can be transmitted via the wireless telecommunication module 18 to a remote device 18c acting as an image recognition server. The image recognition server 18c processes those images and returns back to the device the object detected in the image originally sent, optionally also with location information about the location of the image relative to the camera capturing it. Whilst this can potentially increase the number of objects identifiable, it is subject to a reliable connection between the guidance device 1 and the server 18c, and is affected by lag produced by delays in communication and processing. This can be unacceptable for providing timely warnings and feedback to the user about objects within the environment surrounding the user. In view of this, certain embodiments of the invention may implement an advantageous hybrid method of image recognition where objects detected to be close to the guidance device are subject to a local image recognition routine, whereas objects further from the guidance device are subject to image recognition performed remotely.

Regardless of whether image recognition correctly identifies an object within the environment of the user, it is possible also to detect the presence of unknown objects within the environment of the user, and these can also be used when providing feedback about objects and hazards in the environment. In particular, the guidance device is able to detect the likely distance of such unknown objects from the user and provide feedback and warning of the presence of those objects, how far away they are, their heading (e.g. directly ahead, or either side), and their vertical position (e.g. around head height, directly forward within the path of the user, or near/at ground level). Again, these can be communicated via audio feedback. The depth sensor 17d is particularly useful for accurately determining the distance of objects.

The guidance device 1 also provides tactile feedback about the vertical position and distance of objects detected by the sensor array. Tactile feedback nodules 23, actuated by the vibration generators 46, provide feedback about the vertical position of various obstacles or objects, with each nodule 23 corresponding to a different vertical position.

In particular, the first most-forward tactile feedback nodule, corresponding to the location of a user's index finger, provides warnings about obstacles around head height. The second nodule, corresponding to the location of a user's middle finger, provides warnings about frontal obstacles. The third nodule, corresponding to the location of the ring finger, provides feedback about obstacles around ground level. The fourth nodule, corresponding to the location of the user's little finger, relays custom information, such as information about types of crossings, and their proximity.

Each tactile feedback nodule 23, via the vibration generators 46, changes its vibration characteristics proportionally in response to the proximity of an obstacle within the vertical position that it represents. For example, in the present embodiment, the vibration intensity varies in dependence on proximity, with a high intensity corresponding to a very nearby object (and so the risk of imminent collision), and low intensity corresponding to a faraway object, just within range of the sensor array 17. In alternative embodiments, the tactile or vibration characteristics may be different. For example, a high or low frequency may instead or in combination with intensity, represent nearby or far-away obstacles. Alternatively, pulse width modulation may be used, with a longer "on" duration corresponding to a closer obstacle. The latter is useful to allow the use of less-complex vibration generators 46 that cannot have their frequency or intensity modulated to any great extent.

As mentioned, the memory 13 of the guidance device 1 is limited for the storage of models for the purposes of image recognition. In view of this, and the intended use of the guidance device 1, the recognition of certain objects is prioritised. In particular, one challenge that the present embodiment seeks to overcome is that of navigating crossings.

This includes any type of "people" crossings, preferably including: pedestrian, pelican, zebra, puffin, toucan, level-crossing, bridge, kissing gate, stile and/or unmarked crossings from a dropped kerb. The image recognition routine effectively looks for landmarks based off these crossings e.g. railway barriers, road markings, bumpy pavements, red/green crossing man, black/white pole with yellow flashing orb. The form of crossing will be detected and alert the user as to the type of crossing being approached via audio and/or a haptic feedback sequence. As mentioned, the fourth tactile feedback nodule can be dedicated to the delivery of feedback relating to crossings, in particular their proximity to the user. The dedicated use of a tactile feedback nodule 46 to crossings, particularly road crossings, is very useful for safely navigating an urban environment.

Other categories of objects prioritised for recognition are those that may be encountered during navigation - particular via an urban environment. These can be sub-categorised based on their vertical position, as detected by the cameras of the sensor array 17. The image recognition routine can thus distinguish obstacles in the following categories: o Above: Overhead obstacles such as low hanging tree branches, signs, bushes or anything head level that the user is likely to encounter during their journey o Frontal: Obstacles ahead of the user that they can interact with from shoulder to knee level including: fences, pillars, benches, tables, chairs, bollards, doorways, walls. o Below: Obstacles that are of a different downward/upward gradient than that of the ground, these include: potholes, dipped kerbs, bumpy pavements for crossings, staircases both up and down, ramps etc.

Also as mentioned, the guidance device 1 can be paired with an auxiliary local device 18a, or via a network 18b with a remote device 18c. Doing so can confer a number of functional benefits to the guidance device, some of which have been already described above. Notably, the guidance device 1 can make use of other Internet-enabled services through connections via the network 18b.

One example of this is smart voice interaction, such as those found in the Alexa and Google smart-hub framework. Another is integration of the device 1 with navigation and mapping services (e.g. Google Maps), allowing feedback to the user of voice-based directions and guidance.

Additionally, personal and social Internet-enabled services can be provided and customised. For example, the location of the guidance device 1 can be regularly transmitted via the wireless communication module 18 allowing a user's loved-ones to track their location, if preferred by the user.

These internet-enabled services can be enabled through cellular data connections, for example LTE, 3G, 4G, 5G, etc. To enable access to such cellular data connections, the wireless telecommunication module 18 of the device 1 ideally comprises a Subscriber Identity Module (SIM) card or slot for such card, or other similar means of accessing the cellular data connection - for example, an E-Sim from the hologram network, embedded as part of the guidance device 1.

Although the invention has been described in conjunction with specific embodiments thereof, it is evident that many alternatives, modifications and variations will be apparent to those skilled in the art.

For example, the guidance device may be embodied in alternative forms. The guidance device, rather than being hand-held, can be held on or via another part of the body of the user. For example, the guidance device may be head-mounted. In such an example, the functionality of the guidance device 1 and the auxiliary device 18a (acting as an audio input and output device) of the previously-described embodiments may be combined.

Specifically, an alternative guidance device may be in the form of a pair of glasses, or a device that can be attached to an existing pair of glasses. In such an alternative, the components of the device are repositioned. The sensor array faces forward in use. The weight of the components is evenly distributed for comfortable fitment on the head and face of the user. Tactile feedback sensors may be located at contact points around the head and face. Audio feedback transducers may be integrated in such an embodiment. Input from the user of control and commands can be via touch controls located on, for example, the arms of the glasses, and/or via a microphone as described above, as activated by a "wake-word".

Accordingly, it is intended to embrace all such alternatives, modifications and variations that fall within the scope of the appended claims.