CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority under 35USC§119(e) of US provisional patent application(s) 62/976.876 filed on February 14, 2020, the specification(s) of which being hereby incorporated by reference.

TECHNICAL FIELD OF THE INVENTION

[0002] The present invention relates to the field of navigation in underground mining environments. More particularly, it relates to a position determination system and a corresponding method which leverages the unique patterns created by the numerous rock bolts used for rock support in underground mining environments, to precisely identify a feature of the mine with a know geospatial position and determine a position of a moving entity in the underground mining environment, for example to facilitate navigation therein.

BACKGROUND

[0003] In underground mining environments, the determination of a precise position of a moving entity such as, for example and without being limitative a robot, a transport vehicle, a rock-drilling rig, a loader, or the like, during movement thereof, is often a challenge, given, for example, the inoperability of known navigation systems and methods for surface applications (e.g. the unavailability of Global Positioning System (GPS) signals) in such remote and underground places.

[0004] Hence, in such environments alternative positioning systems including added electrical and/or electronic infrastructure installed inside the underground tunnels of the mines are often provided. However, such alternative positioning systems are usually costly to acquire and to install, Moreover, known alternative positioning systems often tend to suffer from several drawbacks, such as high maintenance fees, need of regular maintenance, etc.

[0005] For example, in an embodiment, a plurality of positioning terminals can be positioned inside an underground mine, in order to communicate with positioning equipment associated with the moving entity and help determining the exact position of the moving entity circulating inside the underground mine. However, such positioning terminals usually necessitate electrical power for functioning and are commonly battery powered, therefore resulting in costly recurrent battery changes to maintain the plurality of positioning terminals operative.

[0006] Alternatively, it is known to use Inertial Measurement Units (IMU), using a combination of local measurements (e.g. acceleration and angular velocity) from on board sensors and specialized algorithms to estimate the position of the moving entity, from an initial known position. However, over time, such IMU loose accuracy due to the accumulation of unbounded errors occurring with each local measurement from the on-board sensors, thereby leading to bias in the position determined using such systems.

[0007] In order to compensate for the unbounded errors, navigation systems for surface applications commonly use complementary positioning systems (e.g. GPS signals) which aids in reducing the effect of accumulating measurement errors through periodical update and correction of the position determined by the IMU, using external position data. In the mining environment, such correction of the position determined by an IMU can be performed using the above-described alternative positioning systems. However, all of the above-mentioned drawbacks associated to the necessity of installing and maintaining the electrical and/or electronic infrastructure of the alternative positioning systems also apply, which is undesirable.

[0008] In view of the above, there is a need for an improved system and method for underground mining environment positioning which, would be able to overcome or at least minimize some of the above-discussed prior art concerns.

SUMMARY OF THE INVENTION

[0009] In accordance with a first general aspect, there is provided a method for underground mining environment positioning of a moving entity. The method comprises the steps of repeatedly: receiving acceleration data relative to movements of the moving entity inside the underground mining environment from an inertial measurement unit (IMU); estimating a current position of the moving entity using the acceleration data from the IMU and a last confirmed position of the moving entity. Simultaneously with the estimation of the position of the moving entity, the method further comprises: generating unique bolt signatures for at least a subset of rock bolts installed on rock walls surrounding the moving entity; searching a bolt map including a tridimensional (3D) position and a unique signature for at least a subset of the rock bolts of the underground mining environment, to detect matches between the generated unique bolt signatures for the rock bolts surrounding the moving entity and the unique signatures of rock bolts of the bolt map; and updating the current position of the moving entity according to the 3D position of a corresponding one of the rock bolts of the bolt map, upon detecting a match between the unique signature of the corresponding one of the rock bolts in the bolt map and a generated unique bolt signature of one of the rock bolts surrounding the moving entity.

[0010] In an embodiment, the step of updating the current position of the moving entity according to the 3D position of the corresponding one of the rock bolts of the bolt map includes updating the estimated current position to match the position of the moving entity with regard to the known 3D of the position of the corresponding one of the rock bolts of the bolt map having the unique signature matching the generated unique bolt signature of the one of the rock bolts surrounding the moving entity.

[0011] In an embodiment, the step of estimating the current position of the moving entity includes determining a distance and a direction of travel of the last confirmed position of the moving entity and updating the last confirmed position of the moving entity using the determined distance and direction of travel of the moving entity.

[0012] In an embodiment, the method further comprises an initial step of initializing the mine and generating the bolt map where at least the subset of the rock bolts installed on the rock walls of the underground mining environment is identified with the 3D position and the unique signature thereof.

[0013] In an embodiment, the step of initializing the mine includes the steps of: recording data relative to the underground mining environment using a data acquisition apparatus recording data while being moved within the underground mining environment, the data relative to the underground mining environment including images of the rock walls of the underground mining environment; processing the recorded data relative to the underground mining environment and generating a point cloud representing the underground mining environment therefrom; performing image processing of the images from the data relative to the underground mining environment and detecting therefrom the rock bolts installed on the rock walls; positioning each one of the detected rock bolts on the point cloud to determine a 3D position thereof; for each one of the detected rock bolts, generating a unique signature using relative positions of the corresponding bolt and a plurality of surrounding rock bolts thereof; and generating the bolt map including the 3D position and the unique signature of each rock bolt.

[0014] In an embodiment, the step of recording data relative to the underground mining environment includes simultaneously recording scanner data of the rock walls surrounding the data acquisition apparatus, capturing images of the rock walls surrounding the data acquisition apparatus, and recording IMU data relative to the position of the data acquisition apparatus.

[0015] In an embodiment, the step of processing the recorded data relative to the underground mining environment and generating a point cloud representing the underground mining environment therefrom includes creating a SLAM point cloud; registering the point cloud; and creating combined image/robot trajectory.

[0016] In an embodiment, the step of performing image processing of the images from the data relative to the underground mining environment and detecting therefrom the rock bolts installed on the rock walls is performed using a machine learning tool trained to detect bolts in images.

[0017] In an embodiment, the machine learning tool is trained using a data set of previously gathered images of rock walls having rock bolts mounted thereon.

[0018] In an embodiment, the unique bolt signature of each one of the rock bolts is generated using a combination of at least one of distances and angles between axes respectively defined between a bolt of interest and a plurality of surrounding bolts and at least one of distances and angles between axes respectively defined between at least a subset of the surrounding bolts. [0019] In an embodiment, the unique bolt signature of each one of the rock bolts is generated using the function: function wherein d1/dtotal, d2/dtotal, d3/dtotal and d4/dtotal each correspond to a relative distance between the bolt of interest and a corresponding one of the surrounding bolts thereof, 11 -2/ltotal, 12-3/ltotal, 13-4/ltotal and 14-1/dtotal each correspond to a relative distance between corresponding surrounding bolts of the bolt of interest and a1-2, a1-2, a2-3, a3-2, a4-3, a3-4, a3-2 and a1-4 correspond to angles between axes respectively defined between the corresponding surrounding bolts and between the bolt of interest the corresponding one of the surrounding bolts.

[0020] In an embodiment, the step of generating unique bolt signatures for at least a subset of rock bolts installed on rock walls surrounding the moving entity comprises the steps of: capturing images of the rock walls surrounding the moving entity during the displacement thereof; performing image processing of the captured images to detect the rock bolts within the images; generating a 3D representation of the detected rock bolts; and generating bolt signatures for at least a subset of the detected rock bolts.

[0021] In accordance with another general aspect, there is provided a computer readable memory having recorded thereon statements and instructions for execution by a computer, said statements and instructions comprising code means for performing the steps of the method as described above.

[0022] A system for underground mining environment positioning of a moving entity. The system comprises a data acquisition apparatus and a computing device. The data acquisition apparatus includes a plurality of cameras disposed to capture photographs of rock walls of the underground mining environment as the data acquisition apparatus is moved therein and an inertial measurement unit (IMU) measuring the acceleration of the data acquisition apparatus and generating acceleration data. The computing device has a processor and a memory and further comprises: a position estimation unit receiving the acceleration data from the IMU and estimating a current position of the moving entity based on the acceleration data and a previous known position of the moving entity; a bolt signature detection unit receiving images of the rock walls surrounding the moving entity from the data acquisition apparatus and generating bolt signatures for at least a subset of the rock bolts shown in the images; a signature matching unit accessing a previously generated bolt map where at least a subset of the rock bolt of the underground mining environment is identified with a tridimensional (3D) position and a unique signature and being in data communication with the bolt signature detection unit, the signature matching unit performing search within the bolt map to identify rock bolts of the bolt maps having the unique signature matching a corresponding one of the bolt signatures of a rock bolt generated by the bolt signature detection unit; and a position determination unit configured to determine the position of the moving entity upon identification of a corresponding rock bolt of the bolt map with the unique signature thereof matching the bolt signature of one of the bolts generated by the bolt signature detection unit, the position of the moving entity being determined according to the 3D position of the corresponding one of the rock bolts of the bolt map.

[0023] In an embodiment, the signature matching unit is in data communication with a database having the previously generated bolt map stored thereon.

[0024] In an embodiment, the position determination unit updates the estimated position of the position estimation unit to match the position of the moving entity with regard to the known 3D position of the corresponding rock bolt of the bolt map with the unique signature thereof matching the bolt signature of one of the bolts generated by the bolt signature detection unit.

[0025] In an embodiment, the bolt signature detection unit is further configured to perform image processing of the images to detect the rock bolts within the images, to generate a 3D representation of the detected rock bolts and to generate the unique bolt signatures for at least a subset of the detected rock bolts.

[0026] In an embodiment, the bolt signature detection unit includes a machine learning tool trained to detect bolts in images. [0027] In an embodiment, the machine learning tool is trained using a data set of previously gathered images of rock walls having rock bolts mounted thereon.

[0028] In an embodiment, the unique bolt signature of each one of the rock bolts is generated using a combination of at least one of distances and angles between axes respectively defined between a bolt of interest and a plurality of surrounding bolts and at least one of distances and angles between axes respectively defined between at least a subset of the surrounding bolts.

[0029] In an embodiment, each one of the unique bolt signature is created using the function: function wherein d1 /dtotal, d2/dtotal, d3/dtotal and d4/dtotal each corresponds to a relative distance between a bolt on interest and a surrounding bolt thereof, 11 -2/ltotal, I2- 3/ltotal, 13-4/ltotal and 14-1 /dtotal each correspond to a relative distance between corresponding surrounding bolts of the bolt on interest and a1-2, a1-2, a2-3, a3-2, a4-3, a3-4, a3-2 and a1-4 correspond to angles between axes respectively defined between the corresponding surrounding bolts and between the bolt of interest the corresponding one of the surrounding bolts.

[0030] In an embodiment, the data acquisition apparatus includes a wall scanner scanning the rock walls of the underground mining environment as the data acquisition apparatus is moved therein.

[0031] In an embodiment, the wall scanner is a LiDAR scanner.

[0032] In an embodiment, the data acquisition apparatus is mounted onto a moveable platform attachable to the moving entity.

BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Other objects, advantages and features will become more apparent upon reading the following non-restrictive description of embodiments thereof, given for the purpose of exemplification only, with reference to the accompanying drawings in which: [0034] Figures 1a to 1c show respectively an isometric view, a longitudinal cross- sectional view and a plan view of a schematic section of a mine showing rock bolts extending inside the rock walls.

[0035] Figure 2 is a partial view of a rock wall of a mine showing the rock bolts inserted therein.

[0036] Figure 3 is a partial view of a rock wall with indication of rock bolts detected thereon, for example using a machine learning tool.

[0037] Figure 4 is a schematic representation of a bolt signature, showing a bolt of interest and four closest surrounding bolts used in defining the bolt signature for the bolt of interest.

[0038] Figure 5 is a schematic representation of distances used in defining a bolt signature, showing a bolt of interest and four closest surrounding bolts used in defining the bolt signature for the bolt of interest, with corresponding distances.

[0039] Figure 6 is a schematic representation of angles used in defining a bolt signature, showing a bolt of interest and four closest surrounding bolts used in defining the bolt signature for the bolt of interest, with corresponding angles therebetween.

[0040] Figure 7 is a flowchart showing the steps for initialization/update of a mine and creation of a corresponding bolt map, in accordance with an embodiment.

[0041] Figure 8 is a flowchart showing the steps for positioning of the moving entity in the underground mining environment, in accordance with an embodiment.

[0042] Figure 9 is a schematic representation of a system for underground mining environment positioning of a moving entity, in accordance with an embodiment.

DETAILED DESCRIPTION

[0043] In the following description, the same numerical references refer to similar elements. The embodiments, geometrical configurations, materials mentioned and/or dimensions shown in the figures or described in the present description are embodiments only, given solely for exemplification purposes.

[0044] Moreover, although the embodiments of the system for underground mining environment positioning and corresponding parts thereof consist of certain configurations as explained and illustrated herein, not all of these components and configurations are essential and thus should not be taken in their restrictive sense. It is to be understood, as also apparent to a person skilled in the art, that other suitable components and cooperation thereinbetween, as well as other suitable configurations, may be used for the system for underground mining environment positioning, as will be briefly explained herein and as can be easily inferred herefrom by a person skilled in the art. Moreover, it will be appreciated that positional descriptions such as “above”, “below”, “left”, “right” and the like should, unless otherwise indicated, be taken in the context of the figures and should not be considered limiting.

[0045] Moreover, although the associated method includes steps as explained and illustrated herein, not all of these steps are essential and thus should not be taken in their restrictive sense. It will be appreciated that the steps of the method for providing underground mining environment positioning described herein may be performed in the described order, or in any suitable order.

[0046] In an embodiment steps of the proposed method are implemented as software instructions and algorithms, stored in computer memory and executed by processors. It should be understood that servers and computers are therefore required to implement to proposed system, and to execute the proposed method. In other words, the skilled reader will readily recognize that steps of various above- described methods can be performed by programmed computers. In view of the above, some embodiments are also intended to cover program storage devices, e.g., digital data storage media, which are machine or computer readable and encode machine-executable or computer-executable programs of instructions, wherein said instructions perform some or all of the steps of said above-described methods. The embodiments are also intended to cover computers programmed to perform said steps of the above-described methods. 10047] To provide a more concise description, some of the quantitative and qualitative expressions given herein may be qualified with the terms "about" and "substantially". It is understood that whether the terms "about" and "substantially" are used explicitly or not, every quantity or qualification given herein is meant to refer to an actual given value or qualification, and it is also meant to refer to the approximation to such given value or qualification that would reasonably be inferred based on the ordinary skill in the art, including approximations due to the experimental and/or measurement conditions for such given value.

[0048] The term “computing device” is used to encompass computers, servers and/or specialized electronic devices which receive, process and/or transmit data. “Computing devices” are generally part of “systems” and include processing means, such as microcontrollers and/or microprocessors, CPUs or are implemented on FPGAs, as examples only. The processing means are used in combination with storage medium, also referred to as “memory” or “storage means”. Storage medium can store instructions, algorithms, rules and/or data to be processed. Storage medium encompasses volatile or non-volatile/persistent memory, such as registers, cache, RAM, flash memory, ROM, as examples only. The type of memory is, of course, chosen according to the desired use, whether it should retain instructions, or temporarily store, retain or update data.

[0049] One skilled in the art will therefore understand that each such computing device typically includes a processor (or multiple processors) that executes program instructions stored in the memory or other non-transitory computer-readable storage medium or device (e.g., solid state storage devices, disk drives, etc.). The various functions, modules, services, units or the like disclosed hereinbelow can be embodied in such program instructions, and/or can be implemented in application- specific circuitry (e.g., ASICs or FPGAs) of the computing devices. Where a computer system includes multiple computing devices, these devices can, but need not, be co-located. In some embodiments, a computer system can be a cloud-based computing system whose processing resources are shared by multiple distinct business entities or other users.

- IQ - [0050] It should be appreciated by those skilled in the art that any block diagrams herein represent conceptual views of illustrative circuitry embodying the principles disclosed herein. Similarly, it will be appreciated that any flow charts and transmission diagrams, and the like, represent various processes which may be substantially represented in computer readable medium and so executed by a computer or processor, whether or not such computer or processor is explicitly shown.

[0051] Referring generally to Figures 1a to 3, one skilled in the art will readily understand that, in the field of underground mines, a large quantity of rock bolts 20 are commonly used for rock mechanics purposes, during the drilling process required for forming a path inside the mine 12. For installation, the rock bolts 20 are inserted into drill holes which are drilled into the rock walls 14 of the mine 12 and secured therein. The drill holes are positioned according to a drill map designed by an engineer and the bolts installed inside the corresponding drill holes thereby define a corresponding bolt pattern. For example, Figures 1a to 1c show respectively an isometric view, a longitudinal cross-sectional view and a plan view of a schematic representation of a section of a mine 12, where rock bolts 20 extend inwardly from an inner surface 14a of the rock wall 14, to provide support to the rock wall 14. Correspondingly, Figure 2 shows an image of a section of a rock wall 14, where rock bolts 20 have been installed and can be seen with a rod section protruding from the inner surface 14a of the rock wall 14.

[0052] Therefore, as can be seen in Figure 2, each rock bolt 20 is visually identifiable on the inner surface 14a of the rock wall 14. In the embodiment shown, each one of the rock bolts 20 includes a protruding rod section having a nut threaded thereon and a metal plate secured against the inner surface 14a of the wall 14 by the nut. In an embodiment, the rock bolts can be used to maintain a mesh 13 against the inner surface 14a of the wall 14.

[0053] Hence, once the rock bolts 20 are installed and together define the above- mentioned bolt pattern, a mapping of the rock bolts 20 can be performed. For example and without being limitative, in an embodiment, the mapping of the rock bolts 20 can be performed by georeferenced imaging or vision of the walls 14 of the mine 12 (i.e. by gathering images or data of the inner surface 14a of the rock wall 14 using an imaging or vision system and a geospatial positioning system determining the position thereof, as the images or data are acquired, therefore allowing the determination of the specific geospatial position of each rock bolt 20 detected from the images or data acquired.

[0054] In view of the above, one skilled in the art will understand that, in an embodiment and as will be described in more details below, specialized software and/or machine learning tools can be used to detect the rock bolts 20 in images of sections of the inner surface 14a of the rock wall 14 and data relative to the image can be used to spatially position the rock bolts 20 inside the underground mining environment. To that effect. Figure 3 shows an example of a portion of an image of the inner surface 14a of a section of a rock wall 14 with indication of rock bolts 20 detected thereon (shown herein as rectangles), by a machine learning tool. One skilled in the art will understand that several types of machine learning could be used for detecting the rock bolts 20 in images of sections of the inner surface 14a of the rock wall 14, such as, for example and without being limitative supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning, transduction, etc.

[0055] Now referring to Figures 4 to 6, it is known by those skilled in the art that, in practice, the pattern designed by the engineer(s) for positioning of the drill holes and the corresponding rock bolts 20 is practically impossible to exactly implement (for example due to the specific work environment). Therefore, in the implemented bolt pattern, each bolt 10 has a unique signature defined by the relative position (i.e. the relative distances and/or angles) of the bolt of interest 20a (i.e. the specific bolt being analyzed and to which the unique signature corresponds) and a subset of surrounding bolts 20b proximate to the bolt of interest 20a (which can include a varying number of rock bolts 20 in the vicinity of the bolt of interest 20a).

[0056] In the course of the present description, the term “unique signature” is used to define a unique identifier defined by the relative positioning of a bolt of interest 20a and a plurality of surrounding bolts 20b (i.e. an identifier which is unique to the pattern of the bolt of interest 20a), such that it can be distinguished from another identifier of other bolt of interests 20a of the underground mining environment.

[0057] Figure 4 shows an exemplary embodiment of a bolt of interest 20a and a subset of surrounding bolts 20b which can be used to define the unique signature of the bolt of interest 20a. As will be described in more details below, in an embodiment, the unique signature of the bolt of interest 20a is defined by the relative positions of the bolt of interest 20a and at least a subset of the surrounding bolts 20b, in 3D. However, for ease of understanding and representation in the course of the present document, the unique signature of the bolt of interest 20a is shown and described herein in 2D. In Figure 4, the bolt of interest 20a is shown with four surrounding bolts 20b cooperating with the bolt of interest 20a to define the unique signature thereof. One skilled in the art will however understand that, in alternative embodiments (not shown) fewer or more than four surrounding bolts 20b can be used to define the unique signature of the bolt of interest 20a. In an embodiment, the amount of surrounding bolts 20b used to define the unique signature of the bolt of interest 20a is selected to provide an advantageous balance between uniqueness of the signature when compared with signatures of other bolts of interests 20a of the underground mining environment, the processing power required to define the unique signatures and subsequently identify the unique signatures and the size of a resulting map including the positions and the unique signatures of bolts 20 of an underground mining environment. For example and without being limitative, in an embodiment, a set of between about three and ten surrounding bolts 20b can be used to define the unique signature of the bolt of interest 20a.

[0058] Referring to Figure 5, there is shown the exemplary embodiment of the bolt of interest 20a and the subset of surrounding bolts 20b, with an example of the distances which can be used in defining the unique signature of the bolt of interest 20a. As can be seen in Figure 5, in the embodiment shown, the distances include distances d1 , d2, d3 and d4, which each represent a distance between the specific bolt of interest 20a and a corresponding one of the surrounding bolts 20b (i.e. the distance between the specific bolt of interest 20a and each one of corresponding boltl , bolt2, bolt3 and bolt4 in Figure 5). In the embodiment shown, the distances also include distances 11-2, I2-3, I3-4 and 14-1 , which each represent a distance between a corresponding one of the surrounding bolts 20b and another surrounding bolt 20b adjacent therewith (i.e. the distance between boltl and bolt2, the distance between bolt2 and bolt3, the distance between bolt3 and bolt4, and the distance between bolt4 and boltl , in Figure 5). One skilled in the art will understand that, in alternative embodiments (not shown) other distances (i.e. different or additional distances) could also be used in defining the unique signature, such as, for example and without being limitative, the distance between bolt2 and bolt4, the distance between boltl and bolt3, in Figure 5. It should be understood that other types of distances between the bolt of interest 20 and the surrounding bolts 20b and/or between the surrounding bolts 20b shown in Figure 5 (or other surrounding bolts not shown in Figure 5) could also be used.

[0059] As will be described in more details below with respect to an instance of a function defining the unique signature, one skilled in the art will understand that, in an embodiment, the distances used to define the unique signature can be relative distances rather than absolute lengths. Indeed, the use of relative distances allows minimization of the processing required in the photogrammetry 3D process for determining the distances from the acquired images and/or additional data, as the calculations required to perform photogrammetry 3D are faster when using relative distances rather than using absolute lengths.

[0060] Now referring to Figure 6, there is shown the exemplary embodiment of the bolt of interest 20a and the subset of surrounding bolts 20b, with an example of the angles which can be used in defining the unique signature of the bolt of interest 20a. As can be seen in Figure 6, in the embodiment shown, the angles include angles a1- 2, a1-2, a2-3, a3-2, a4-3, a3-4, a3-2, a1-4, which each represents an angle between two axes respectively defined between the corresponding one of the surrounding bolts 20b and the bolt of interest 20a and between the corresponding one of the surrounding bolts 20b and another surrounding bolt 20b adjacent therewith. One skilled in the art will again understand that, in alternative embodiments (not shown) other angles could also be used in defining the unique signature, such as, for example and without being limitative, other angles between surrounding bolts 20b than those shown in Figure 6. [0061] In view of the above, it is understood that, in an embodiment, the unique signature can be generated through a combination of specific characteristics stemming from the geospatial relations of the bolt of interest 20a and N closest surrounding bolts 20b, to create a unique identifier that can specifically identify a particular bolt 20 in a map of the plurality of bolts 20.

[0062] In an embodiment, the unique signature can be generated by a function such as, for example and without being limitative the function below, which returns a unique ID. In the function below, the reference identifiers d1 , d2, d3, d4, 11-2, I2-3, I3- 4, 14-1 , a1-2, a1-2, a2-3, a3-2, a4-3, a3-4, a3-2 and a1-4 correspond to the equivalent distances or angles as defined in Figures 5 and 6 and discussed above. The identifiers dtotal and Itotal correspond respectively to the total of d1 , d2, d3 and d4 and the total of 11-2, I2-3, I3-4, 14-1 , such that the function uses relative distances rather than lengths.

[0063] Similarly to a hashing function, the function defined above returns a unique signature for a bolt of interest 20a. One skilled in the art will understand that, in an embodiment, the distances and angles used as input for the function defining the unique signature can be rounded using a magnitude within the range of error of the data acquisition apparatus, to compensate for possible recording errors of the data acquisition apparatus. Once again, it will be understood that, in alternative embodiments (not shown), the function used to define the unique signature could have more or fewer parameters, or different parameters corresponding to the distances and angles used for defining the unique signature.

[0064] Indeed, in an embodiment, other featured of the bolt of interest 20a could also be used in defining the unique signature thereof. For example and without being limitative, in an embodiment the type of rock bolt could be used as a parameter in the unique signature of the rock bolt. For example and without being limitative, in another embodiment the orientation of the rock bolt could also be used as a parameter in the unique signature of the rock bolt. Method for underground mining environment positioning using unique rock bolts signatures

[0065] In view of the above, as will be described in more details below, the signature of each rock bolt 20 (as defined by the relative position of the bolt of interest 20a and a subset of surrounding bolts 20b) can therefore be used in a 3-dimensional environment, to define the exact positioning of a moving entity inside the mine.

[0066] Referring now to Figures 7 and 8, a method which allow underground mining environment positioning using unique rock bolts signatures is shown in accordance with an embodiment.

[0067] Figure 7 shows the steps for initializing the mine and generating a bolt map where the rock bolts 20 of the mine are identified with a 3D position (in X, Y, Z) and a unique signature. One skilled in the art will also understand that, in an embodiment, the steps shown in Figure 7 for initializing the mine can also be repeatedly performed after the first initialization, to update the bolt map where each rock bolt 20 of the mine is identified with a 3D position (in C,U,Z) and a unique signature, in order to ensure that the bolt map always corresponds to the current condition of the mine. One skilled in the art will understand that, in an alternative embodiment, only a subset of the rock bolts 20 of the mine can be identified with a 3D position (in C,U,Z) and a unique signature in the bolt map.

[0068] Figure 7 shows the general starting step 100 for initializing/updating the mine and the sequence of sub steps to be performed for performing the initialization/update of the mine and generating the bolt map, in accordance with an embodiment.

[0069] In the embodiment shown, the sub steps include the recording step 102 of recording data relative to the underground mining environment. In an embodiment, the step 102 of recording data relative to the underground mining environment can include simultaneously recording scanner data, and capturing images of the rock walls, along with inertial motion unit (IMU) data, which together allow defining the underground mining environment and the corresponding geospatial position simultaneously. In the course of the present description, a LiDAR is described as the scanner being used, but one skilled in the art will understand that, in alternative embodiment, other types of scanner capable of generating point clouds could be used. Hence, in an embodiment, this step is performed using a data acquisition apparatus having the necessary equipment for recording data and capturing images while being moved within the underground mining environment.

[0070] In an embodiment, this step can be performed, for example by moving a data acquisition apparatus along the path of the mine, with the data acquisition apparatus including simultaneously operating LiDAR, cameras and IMU all operating as the data acquisition apparatus is moved inside the mine. It will be understood that the combination of a LiDAR, cameras and an IMU can generate data relative to the geographic position of the data acquisition apparatus, while simultaneously performing 3D laser scanning of the environment of the data acquisition apparatus, and image capture of the environment of the data acquisition apparatus. Operational details of a data acquisition apparatus in accordance with an embodiment, will be described in more details below.

[0071] Once the data is acquired, the recording data relative to the underground mining environment gathered by the data acquisition apparatus can be processed to analyze and map the visible features of the rock walls 14 of the mine 12. Hence, in an embodiment, the method can include the subsequent steps of generating a point cloud representing the underground mining environment 104 and detecting the bolts on the images gathered by the data acquisition apparatus 106.

[0072] In an embodiment, the step of generating a point cloud representing the underground mining environment 104 includes creating a SLAM point cloud 104a, registering the point cloud 104b and creating combined image/robot trajectory 104c, but one skilled in the art will understand that, in an alternative embodiment (not shown), other alternative steps could be provided to generate an accurate point cloud.

[0073] One skilled in the art will understand that the step of detecting the bolts on the images gathered by the data acquisition apparatus 106 can be performed, for example and without being limitative, using a machine learning tool trained to detect bolts in images. For example and without being limitative, the machine learning tool could be trained using a data set of previously gathered images of rock walls having rock bolts mounted thereon.

[0074] Once the point cloud has been generated and the bolts have been identified in the images gathered by the data acquisition apparatus, a further step of positioning each bolt on the point cloud 108 can be performed. The step of positioning each bolt on the point cloud 108 therefore allows to determine the 3D position (i.e. the X, Y and Z position) of each bolt within the underground mining environment.

[0075] Finally, using the 3D position of a bolt of interest and the position of N-Closest surrounding bolts, the unique signature of the bolt of interest can be determined and generated 110, for example using the function described above, and the unique signature can be associated with the bolt. This step can be performed specifically for each one of the rock bolts 20 (i.e. using each one of the rock bolts 20 as the bolt of interest 20a to generate its unique signature) to associate a unique signature to each bolt. One skilled in the art will again understand that, in an alternative embodiment, this step can be performed only for a subset of the rock bolts 20 of the mine 12.

[0076] In view of the above, it will be understood that, in an embodiment, the generated bolt map therefore includes the sum of the combined position and unique signature of each rock bolt 20 of the underground mining environment. In an embodiment, the combined position and unique signature of each bolt can be compiled and stored in a bolt map stored in a dynamic database of a data server.

[0077] One skilled in the art will understand that the above-described steps as shown in Figure 7 for initializing/updating the mine and generating the bolt map will therefore require important scanning time and resources and computation time and resources. Hence, it will be understood that it is therefore desirable to perform the initialization/update of the mine and generation of the bolt map, before positioning and/or navigation of a moving entity inside the mine using the rock bolts can be performed. [0078] Now referring to Figure 8, once initialization/update of the mine and generation of the bolt map 100 has been performed, positioning and/or navigation of a moving entity inside the mine can be realized.

[0079] Figure 8, therefore shows the general starting step 200 for positioning and/or navigation of a moving entity inside the mine and the sequence of sub steps to be performed for performing the positioning and/or navigation of the moving entity inside the mine 12, in accordance with an embodiment.

[0080] In the embodiment shown, the method includes the steps of recording acceleration data from an IMU 204. From the acquired data of the IMU, the method includes the step 205 of performing estimation of the position of the moving entity, using the combination of the acceleration data of the IMU and a last confirmed position of the moving entity.

[0081] During the step of performing estimation of the position of the moving entity 205, an approximative position is continuously or repeatedly updated based on a distance covered from the last known position, as defined by the acceleration data gathered from the IMU moving along with the moving entity, which is transmitted to a computing device and is processed to determine a distance travelled from the last known position and a direction of travel. In other words, in an embodiment, this step includes determining a distance and a direction of travel from the last confirmed position of the moving entity and updating the last confirmed position of the moving entity using the determined distance and direction of travel of the moving entity.

[0082] Still referring to Figure 8, the method for positioning and/or navigation of a moving entity inside the mine further includes detecting bolt signatures for bolts surrounding the moving entity 202. The step of detecting bolt signatures for bolts surrounding the moving entity 202 is performed simultaneously with the estimation of the position using the acceleration data of the IMU and the last confirmed position of the moving entity 205.

[0083] As will be described in more details below, combining the estimation of a position of the moving entity using data recorded from the IMU and processed by a computing device with the positioning using the bolt signatures for bolts surrounding the moving entity, allows a determination of an accurate position which alleviates the bias of the positioning using only the data from the IMU, while reducing the search time and providing faster results as compared to positioning using only the bolt signatures for bolts surrounding the moving entity .

[0084] As can be seen in Figure 8, in the embodiment shown, the general step of detecting bolt signatures for bolts surrounding the moving entity 202 includes the steps of capturing images of the rock walls surrounding the moving entity during the displacement thereof 202a, performing analysis of the images to detect the rock bolts within the images 202b, generating a 3D representation of the detected rock bolts 202c and finally generating bolt signatures for at least a subset of the detected rock bolts 202d.

[0085] For example and without being limitative, in an embodiment, when recording images of the rock walls surrounding the moving entity during the displacement thereof 202a, the images can be recorded using the cameras of the above- mentioned data acquisition apparatus mounted to or being pulled/pushed/carried by the moving entity. In an embodiment, the images can be captured with overlap therebetween in order to allow usage of photogrammetric parallax in subsequent image analysis for the determination of the position of the rock bolts 20 surrounding the moving entity (when the images were captured) and to determine the unique signatures of the rock bolts 20, in real time (or near-real time). One skilled in the art will however understand that, in alternative embodiments other methods such as, for example and without being limitative, online simultaneous localization and mapping (SLAM) and pictures could rather be used to define the unique signatures of the rock bolts 20 in real time (or near-real time).

[0086] Similarly to the rock bolt detection in images performed during initialization/update of the mine, the analysis of the images to detect the rock bolts 20 within the images 202b can be performed, for example and without being limitative using a machine learning tool trained to detect rock bolts 20 in images. Once again, for example and without being limitative, the machine learning tool could be trained using a data set of previously gathered images of rock walls 14 having rock bolts 20 mounted thereon. [0087] The sub step of generating a 3D representation of the detected rock bolts 202c can be performed using photogrammetry (i.e. by determining the position of the rock bolts 20 in 3D using measuring and interpretation of the photographic images gathered at step 202a). Once a 3D position of each rock bolt 20 shown in the captured images has been determined, the 3D representation of the detected rock bolts 20 can be generated by representing the bolts 20 according to their 3D position. Once again, specialized software and/or machine learning tools can be used to perform the photogrammetry of the gathered images and the generation of the 3D representation of the detected rock bolts 20.

[0088] The generation of the bolt signature for at least a subset of the detected rock bolts 202d, is once again performed as described above, using the 3D position of each bolt of interest and the position of N closest surrounding bolts, to determine the unique signature of each bolt of interest, for example using the function described above.

[0089] As mentioned above, one skilled in the art will understand that, theoretically, determination of the position of the moving entity could be performed based only on determination of the distance covered from a known position using acceleration data from an IMU. However, in practice, when using data from an IMU, the accumulated errors, for example caused by deviations of the moving entity from a straight displacement line, often results in significant drift in positioning and consequent lost of position. However, as will be described in more details below, combining determination of the distance covered from a known position using data from an IMU with detection of bolt signatures for rock bolts 20 surrounding the moving entity makes it possible to reset the position of the moving entity to a known position each time a signature is identified on the bolt map. On the other hand, when no signature is identified on the bolt map (for any reason), the IMU can keep track of the position to still provide an estimated position, while waiting for a new identification of a bolt signature on the bolt map and consequent position determination.

[0090] In an embodiment, the estimated position can also be used to reduce the search of a signature of the rock bolts 20 surrounding the moving entity to a specific sector of the bolt map corresponding to the sector associated with the estimated position of the moving entity, therefore minimizing the computing time required to find a match between the bolt signature of a rock bolt 20 surrounding the moving entity and a rock bolt 20 of the bolt map.

[0091] In view of the above, following the step of generation of the bolt signatures for at least a subset of the detected rock bolts 202d, the method for positioning and/or navigation of a moving entity inside the mine includes the further step of searching bolt signatures corresponding to the generated bolt signatures for at least a subset of the detected rock bolts 20 in the previously generated bolt map 206.

[0092] In an embodiment, to perform the search of bolt signatures corresponding to the generated bolt signatures in the previously generated bolt map 206, two options are available depending on whether an estimated position of the moving entity is available or not.

[0093] In the alternative where an estimated position of the moving entity is available, searching bolt signatures corresponding to the generated bolt signatures for at least a subset of the detected rock bolts can be restricted to a section of the bolt map corresponding to the estimated position of the moving entity 206a, 206b (i.e. limiting the search in the bolt maps to rock bolts located within a predetermined perimeter around the estimated position of the moving entity).

[0094] In the alternative where an estimated position of the moving entity is unavailable, searching bolt signatures corresponding to the generated bolt signatures for at least a subset of the detected rock bolts can be performed without being restricted to a section of the bolt map corresponding to the estimated position of the moving entity (i.e. be performed on the entire bolt map) 206b. Once again, specialized software and/or machine learning tools can be used to perform the search of bolt signatures corresponding to the generated bolt signatures for at least a subset of the detected rock bolts on the bolt map at step 206b.

[0095] If the results of the steps of searching the bolt signatures corresponding to the generated bolt signatures for at least a subset of the detected rock bolts on the bolt map 206a and/or206b are negative, a new iteration can be performed, without updating the position of the moving entity to a confirmed known position (i.e. the position only being estimated based on the data from the IMU from the last known position, while waiting for a new identification of a bolt signature on the bolt map and the corresponding position determination).

[0096] If the results of the steps of searching the bolt signatures corresponding to the generated bolt signatures for at least a subset of the detected rock bolts on the bolt map 206a and/or 206b are positive (i.e. a bolt signature corresponding to the generated bolt signatures generated using pictures acquired by the data acquisition system of the moving entity match a bolt signature of the bolt map), the estimated position of the moving entity and the actual position of the moving entity can be updated 208 to match the position of the moving entity with regard to the known position of the bolt of interest for which the unique signature has been identified 208. Once the position of the moving entity has been determined, a new iteration can be performed.

[0097] As previously mentioned, this process is iterative and can be continuously or repeatedly performed during the displacement of the moving entity inside the mine to provide continuous or repeated estimated or validated position of the moving entity. For example and without being limitative, the position of the moving entity can be displayed on a display device to a user of the moving entity or can be transmitted to a control center which can operate to use the positioning data of the moving entity, for example to coordinate the movement of moving entities inside the mine, to control the movement of the entities inside the mine, or the like.

System for underground mining environment positioning using unique rock bolts signatures

[0098] In view of the above-described method for positioning in underground mining environment using rock bolt signatures, one skilled in the art will understand that a system 10 having the required components for performing the steps of the method and allow positioning and/or navigation of a moving entity inside the mine can also be provided. One skilled in the art will understand that elements mentioned above, and which apply to the system for positioning in underground mining environment using rock bolt signatures, will not be repeated herein but should be considered as being part of the disclosure regarding the system 10. [0099] Referring to Figure 9, in an embodiment, the system 10 includes a data acquisition apparatus 30 including a plurality of cameras 32 disposed to capture images of the rock walls 20 of the underground mining environment as the data acquisition apparatus 30 is moved therein. For examples and without being limitative, in an embodiment, the cameras 32 are disposed to capture images of the rock walls of the underground mining environment over about 270 degrees, from a lateral surface of the data acquisition apparatus 30, as the data acquisition apparatus 30 is moved along a path of the underground mining environment. The data acquisition apparatus 30 also includes an IMU 34 including the required components such as, for example and without being limitative, accelerometers, gyroscopes and the like, to measure the acceleration of the data acquisition apparatus 30 in 3D and to generate acceleration data representative of the acceleration thereof. In an embodiment, the data acquisition apparatus 30 can also include a wall scanner 36, such as, for example and without being limitative, a LiDAR, scanning the rock walls of the underground mining environment as the data acquisition apparatus 30 is moved therein. For example and without being limitative, in an embodiment, the data acquisition apparatus 30 can be mounted onto a moveable platform attachable to the moving entity. In an embodiment, the moveable platform can be a trailer being attached behind the moving entity and being pulled by the moving entity as it moves within the mine.

[00100] The system 10 further comprises a computing device 40 having a processor 42 for processing instructions and data and a memory 44 storing instructions and/or data acquired by the data acquisition apparatus 30.

[00101] In an embodiment, the computing device 40 includes a position estimation unit 50 receiving the acceleration data from the IMU 34 of the data acquisition apparatus 30, processing the acceleration data from the IMU 34 and estimating the position of the moving entity based on the acceleration data and a previous known position. In an embodiment, the position estimation unit 50 estimates the position of the moving entity by determining a distance travelled from a known position by the moving entity from the acceleration data received from the IMU 34 and adjusting the position from the last known position using the distance travelled therefrom. [00102] In an embodiment, the computing device 40 also includes a bolt signature detection unit 52 configured to receive images of the rock walls surrounding the moving entity from the data acquisition apparatus 30, to perform image processing of the received images to detect the rock bolts and to generate bolt signatures for at least a subset of the bolts shown in the images. In more details, in an embodiment, the bolt signature detection unit 52 is configured to perform image processing of the received images to detect the rock bolts within the images, to generate a 3D representation of the detected rock bolts and to generate unique bolt signatures for at least a subset of the detected rock bolts, for example using the above described function:

[00103] In an embodiment, the computing device 40 further includes a signature matching unit 54 in data communication with the bolt signature detection unit 52 and with a database comprising a previously generated bolt map 60 where the rock bolts of the mine are identified with a 3D position (in X, Y, Z) and a unique signature. The signature matching unit 54 is configured to query the database and search the database to perform signature matching between bolt signatures of the bolt map stored in the database 60 and the bolt signatures generated by the signature detection unit 52, to identify bolts of the bolt maps having a unique signature matching a signature of a rock bolt 20 generated by the bolt signature detection unit 52. One skilled in the art will understand that, in an alternative embodiment, the bolt map could be stored in a memory of the computing device 40, such that signature matching can be performed directly from the bold map stored in the memory of the computing device 40 and the signatures of the rock bolts 20 generated by the bolt signature detection unit 52.

[00104] In an embodiment, the computing device 40 further includes a position determination unit 56 configured to determine the position of the moving entity upon identification of a match between at least one rock bolt of the bolt maps having a unique signature and a signature of a rock bolt generated by the bolt signature detection unit 52 and updating the estimated position of the moving entity generated by the position estimation unit, to match the position of the moving entity with regard to the known position of the bolt of interest for which the unique signature has been identified as a match with the signature of the rock bolt generated by the bolt signature detection unit 52 .

[00105] The position of the moving entity is determined by the position determination unit 56, based on the position of the specific identified rock bolt 20 in the bolt map (each rock bolt having a tridimensional position associated therewith in the bolt map) and the position of the data acquisition apparatus 30 respective to the specific rock bolt when capturing the images showing the specific bolt. In order to do so, in an embodiment, the images can be captured with overlap therebetween in order to allow usage of photogrammetric parallax in subsequent image analysis for the determination of the position of the entity with regard to a specific rock bolt surrounding the moving entity when the images were captured.

[00106] Several alternative embodiments and examples have been described and illustrated herein. The embodiments of the invention described above are intended to be exemplary only. A person of ordinary skill in the art would appreciate the features of the individual embodiments, and the possible combinations and variations of the components. A person of ordinary skill in the art would further appreciate that any of the embodiments could be provided in any combination with the other embodiments disclosed herein. It is understood that the invention could be embodied in other specific forms without departing from the central characteristics thereof. The present examples and embodiments, therefore, are to be considered in all respects as illustrative and not restrictive, and the invention is not to be limited to the details given herein. Accordingly, while the specific embodiments have been illustrated and described, numerous modifications come to mind. The scope of the invention is therefore intended to be limited solely by the scope of the appended claims.