Field of the Invention

The present invention relates to an apparatus and method for obtaining a 3D profile of train wagons used for transporting ore and relates particularly, though not exclusively, to such an apparatus and method for measuring the wagon profile prior to loading and the wagon volume during the loading process and upon completion of the loading process.

Background to the Invention The cost of transporting of iron ore and coal products from mine site to port represents a significant portion of the production cost.

Spillage resulting from overloaded wagons may cause environmental damage, damage to rails and rolling stock and is often subject to fines by the carrier. Under-loaded wagons will reduce the train capacity and increase fuel cost due to higher aerodynamic drag. Optimising train wagon loads, by avoidance of both overloading and under-loading, improves the transport efficiency and offers a significant saving in transport cost.

Therefore during the loading of ore into wagons of a train used for transporting the ore from mine site to port, it is important that an accurate measurement of wagon loading is made. This is important in order to avoid both overloading and under-loading of train wagons.

The majority of wagon loaders are batch or choke feed types:

• Batch Loaders

Batch loaders fill the train wagon by dumping one or more weighed batches of ore into the wagon positioned below a batch hopper. Multiple batches from separate hoppers are often employed in order to provide improved control of the loading rate. • Choke Feed Loaders

Choke feed loaders operate by positioning the discharge chute close to the top of the wagon and filling the wagon up to the chute level. Once the wagon level reaches the discharge chute, then material flow will be choked. The discharge gates are closed as the wagon passes the edge of the discharge chute. The choke feeder loading process is illustrated in Figure !

A common problem with both loading systems is that the loading rate is difficult to control due to variation in material flow characteristics at the bin discharge gate. Products that flow freely will fill the wagon too rapidly and splash out of the wagon. In the extreme case, the discharged material may become fluidised and flow freely over the top of the wagon. Other products may bind at the discharge gate and require a large gate opening to release product from the discharge hopper. Existing wagon loading control systems use various gate sequencing and position modulation schemes in an attempt to overcome the product flow variance. This usually involves a large initial opening to release the product, followed by closing to a pre-set filling position. Fully automatic operation is not achieved and an operator is required to monitor the loading process and alter the loading control setting when problems- are noticed.

Current wagon loading control systems also use a loaded wagon profiler to provide feedback of the final load profile to an operator or directly to the wagon loading control system in order to tune the gate sequencing parameters. A train may be assembled from multiple wagon types, where each wagon type has different dimensions of length, width, height and shape. Wagon loading systems require the wagon type to be identified in order to control the loading process. The wagon type is usually identified via an RFID (Radio- frequency identification device) tag attached to the wagon. The RFID tag is read as the wagon enters the loading facility and used to lookup the wagon details in a wagon database. Existing wagon profilers employ 2D laser scanners to profile a cross-section of the wagon load and then use the train forward motion to move the cross- section along the wagon to provide the third dimension of the profile. The reliance on train forward motion to provide the third dimension dictates that the positional accuracy of the cross-section along the length of the wagon depends on the accuracy of an externally supplied train speed signal. This introduces considerable latency and places a limitation on the acquisition speed and accuracy of the scanned profile.

Existing wagon loading facilities use the RFID tag wagon type identification as the forward motion and the inaccuracy of train speed measurement limits the use of 2D scanners for the measurement of the wagon dimensions.

Furthermore, existing 2D scanners are not able to measure the load profile in the immediate vicinity of the discharge chute of a train load-out bin. This is due to the flow of product on each side of the laser cross-sectional scan line. In addition, an important factor for the determination of correct wagon loading is the load profile just inside the leading and trailing edges of a wagon. The beam divergence and vertical orientation of a 2D scanner means that the profile at the leading and trailing edges may lie in the shadow of the wagon edge. This means that actual load profile may be different from that detected by the 2D scanner.

Existing wagon loading control systems use an open loop implied rate control, that is, the loading rate is based on a rate implied from the gate opening and a set of gate control parameters rather than on dynamic measurement of the wagon load . The present invention was developed with a view to providing wagon dimension information prior to loading, dynamic measurement of the wagon load during the loading process and/or an accurate load profile of loaded train wagons, using a 3D camera to scan the contents of the wagons. References to prior art in this specification are provided for illustrative purposes only and are not to be taken as an admission that such prior art is part of the common general knowledge in Australia or elsewhere.

Summary of the Invention According to one aspect of the present invention there is provided a 3D profile apparatus for train wagons, the apparatus comprising: a 3D image sensor mounted above a rail track on which the train wagons ride and adapted to provide 3D images of a wagon below; and, a data processing apparatus for processing the 3D images produced by the 3D image sensor to generate a 3D profile of the train wagon whereby in use, dynamic measurement of a wagon load during a loading process and an accurate load profile of loaded train wagons can be obtained.

Preferably the 3D image sensor is one of a plurality of 3D image sensors mounted above the rail track to provide 3D images of the wagon below, whereby in use, when the individual images are fused together they provide a complete panoramic three dimensional picture of the wagon profile. Typically the 3D image sensors are located along a longitudinal centre line of the wagon and inside a leading edge and a trailing edge of the wagon so that the whole inside surface of the wagon is within the image sensors' field of view. In one embodiment the 3D image sensors are 3D time-of-flight cameras which measure the distance to an object in front of the camera by analysing the time for a light pulse to travel from an illumination source to the object and back.

Preferably the data processing apparatus comprises a wagon profile recorder and a wagon profile fill controller. The data processing apparatus is also typically connected to a wagon database. Preferably two wagon profile 3D image sensors and two load profile 3D image sensors are connected to the wagon profile recorder, and two fill profile 3D image sensors are connected to the wagon profile fill controller. Typically the train is an ore train and train wagons are loaded with ore. In a preferred embodiment the 3D profile of the train wagon includes a load profile of the wagon when it is being loaded with ore. Advantageously the 3D image sensors are all located a prescribed height above the rail track angled such that the interior of the wagon is within the image sensors fields of view.

According to another aspect of the present invention there is provided a 3D profile method for train wagons, the method comprising the steps of: obtaining 3D images of a train wagon from a 3D image sensor mounted above a rail track on which the train wagons ride; and, processing the 3D images to generate a 3D profile of the train wagon whereby in use, an accurate profile of wagons prior to loading, dynamic measurement of a wagon load during a loading process and an accurate load profile of loaded train wagons can be obtained . Preferably the step of processing the 3D images involves transforming the 3D images in order to express the 3D wagon profile in terms of a track coordinate system. Typically the track coordinate system uses a local (right- hand) Cartesian coordinate system (x, y, and z) to define the wagon profile, with the wagon profile X and Y axis defined with reference to a track reference position, and the wagon profile Z axis is defined with reference to a track reference level. Preferably the X axis is aligned with the rail track, the Y axis is perpendicular to the rail track, and the Z axis is orthogonal to the X and Y axes with the positive direction upwards towards the image sensors.

Normally, the position and orientation of the image sensor is fixed in relation to the track coordinate origin. Typically the 3D image sensor is one of a plurality of 3D image sensors mounted above the rail track to provide 3D images of the wagon below, and the step of processing the 3D images involves fusing the individual images together to provide a complete panoramic three dimensional picture of the wagon profile. Preferably the position and orientation of each image sensor is accurately measured to define the optical axis of each image sensor in relation to the track coordinate system.

Preferably target point data from the plurality of 3D image sensors is combined to create a wagon profile expressed in terms of the track coordinate system. Advantageously each image sensor is capable of providing target point data at a high frame rate (typically up to 60 frames per second), with an image capture time of less than ten milliseconds. Preferably the high frame rate provides the ability to present a continuously updated wagon profile. The fast image capture provides for accurate capture whilst the wagon is moving.

Preferably the target points from all image sensors are mapped into a wagon profile point map in order to provide a plan view profile of the wagon. Typically the wagon profile is a two dimensional height map, one dimension of the array extending parallel to the rail direction whilst the second dimension extends perpendicular to the rail direction. The number of elements in each dimension is typically selected to match the available resolution of the image sensors. The height value in each array element represents the third dimension and is expressed as a height above the track coordinate reference level. Typically the step of processing the 3D images to generate a 3D profile of the train wagon includes the step of generating a 3D profile of a wagon after it has been filled, as well as the step of generating a 3D profile of a wagon while it is being filled. Preferably the wagon profile prior to loading is used to identify a wagon type, dimensions and capacity. Preferably the step of generating a 3D profile of a wagon while it is being filled includes culling the image points of a wagon model from the 3D profile of the wagon to generate a dynamically varying load profile.

Advantageously the dynamically varying load profile is used together with the wagon type, dimensions and capacity to determine a dynamically varying wagon load volume. Preferably the dynamically varying wagon load volume is used to calculate an actual fill volume rate. Preferably the actual fill volume rate is compared to a desired fill volume rate curve to control the discharge of material into the wagon.

Throughout the specification, unless the context requires otherwise, the word "comprise" or variations such as "comprises" or "comprising", will be understood to imply the inclusion of a stated integer or group of integers but not the exclusion of any other integer or group of integers. Likewise the word "preferably" or variations such as "preferred", will be understood to imply that a stated integer or group of integers is desirable but not essential to the working of the invention.

Brief Description of the Drawings

The nature of the invention will be better understood from the following detailed description of a specific embodiment of the 3D profile apparatus and method for train wagons, given by way of example only, with reference to the accompanying drawings, in which:

Figures 1A, 1 B and 1C illustrate a prior art choke feeder loading process;

Figure 2 is a side view of one embodiment of the 3D profile apparatus according to the present invention;

Figure 3 illustrates the process of obtaining a 2D height map for each camera using the apparatus of Figure 2;

Figure 4 illustrates the camera coordinates as employed in the apparatus of Figure 2;

Figure 5 illustrates the camera target coordinates as employed in the apparatus of Figure 2;

Figures 6a and 6b are a process flow diagram illustrating the operation of one embodiment of a profile recorder in the 3D profile method according to the present invention; Figure 7 is a process flow diagram illustrating the operation of one embodiment of a fill controller in the 3D profile method according to the present invention; and,

Figure 8 is a process diagram showing a preferred method of comparing the actual measured volume rate and fill volume profile to control the fill rate in one embodiment of the 3D profile method according to the present invention.

Detailed Description of Preferred Embodiments

A preferred embodiment of a 3D profile apparatus 0 for train wagons 12, in accordance with the invention, as illustrated in Figure 2, comprises a 3D image sensor 14 mounted above a rail track 16 on which the train wagons 12 ride. The 3D image sensor is one of a plurality of 3D image sensors 14 mounted above the rail track 16 to provide 3D images of a wagon 12 below, and when the individual images are fused together they provide a complete panoramic three dimensional picture of a wagon profile.

The 3D profile apparatus 10 also comprises a data processing apparatus 20 for processing the 3D images produced by the 3D image sensors 14 to generate a 3D profile of the train wagon 12. In this embodiment the data processing apparatus 20 comprises a 3D wagon profile recorder 26 and a 3D wagon profile fill controller 28. The data processing apparatus 20 is also typically connected to a wagon database 32. Preferably two load profile 3D image sensors 14a and 14b, connected to the 3D wagon profile recorder 26, are located along a longitudinal centre line of a loaded wagon 12a, and inside a leading edge 22a and a trailing edge 24a of the wagon 12a respectively, so that the whole inside surface of the loaded wagon 12a is within the image sensors' fields of view. This provides an ideal position to measure the load profile level in relation to the top edge of the wagon. Preferably two fill profile 3D image sensors 14c and 14d, connected to the 3D wagon profile fill controller 28, are also located along a longitudinal centre line of a wagon 12b in the process of being batch loaded. The image sensors 14c and 14d are located outside the trailing edge 24b and inside the trailing edge 24b of the wagon 12b respectively, so that the rear inside surface of the loaded wagon 12b is within the image sensors' fields of view, as shown in Figure 2. This provides an ideal position to measure the profile level of the ore as it is being loaded into the wagon 12b, in relation to the top edge of the wagon. Preferably two wagon profile 3D image sensors 14e and 14f, also connected to the 3D wagon profile recorder 26, are located along a longitudinal centre line of an empty wagon 12c prior to filling, and inside a leading edge 22c and a trailing edge 24c of the empty wagon 12c respectively, so that the whole inside surface of the empty wagon 12c is within the image sensors' fields of view. This provides an ideal position to measure the profile of the wagon.

In the described embodiment the 3D image sensors are 3D time-of-flight cameras 14 which measure the distance to an object in front of the camera by analysing the time for a light pulse to travel from an illumination source to the object and back. The 3D time-of-flight cameras 14 have a short image capture time (<10mS) which provides an instantaneous blur free image, independent of the train speed during loading. Triggering of the cameras 14 to capture the wagon profile may be at predetermined time intervals. Alternatively, they may be triggered by a remote signal or by the position of the wagon 12 as determined by the leading or trailing edges of the wagon. The 3D cameras 14 are able to capture the wagon profile at high frame rates to provide a continuously updated 3D wagon profile.

In the illustrated embodiment the train is an ore train and the train wagons 12 are loaded with ore. The 3D profile of the train wagons includes a load profile of the wagon 2a when it is filled with ore, dynamic measurement of a load in the wagon 12b during the loading process, as well as a wagon profile of the empty wagon 12c prior to filling. Preferably the 3D cameras 14a, 14b, 14e and 14f are located a prescribed height above the rail track 16 facing directly downwards. Preferably the 3D cameras 14c and 14d are located at prescribed angles in relation to the top edge of the wagon 12b so that the whole rear inside surface of the loaded wagon 12b is within the cameras' fields of view.

A preferred 3D profile method, using the apparatus of Figure 2, will now be described in detail with reference to Figures 2 to 8. The processes illustrated in the accompanying drawings employ two 3D image sensors (cameras 14a and 14b, cameras 14c and 14d, or cameras 14e and 14f, respectively). A similar process may be performed using only a single image sensor above each wagon. A plurality of image sensors provides improved image acquisition of the full wagon profile.

The 3D profile method typically comprises both recording the load profile of a filled wagon 12a, controlling the loading of an empty wagon 12b and recording the wagon profile of an empty wagon 12c. Figures 6a and 6b illustrate the process involved in recording the wagon profile of an empty wagon 12c or the load profile of the filled wagon 12a, and Figure 7 illustrates the process involved in controlling the loading of an empty wagon 12b. There is some overlap in the two processes, particularly in the image acquisition methodology, and therefore the same reference numerals are employed in Figures 6 and 7 to refer to the similar process steps. The first step 100 of obtaining 3D images of the wagon 12 from the 3D cameras 14 occurs in relation to each of the wagons 12a, 12b and 12c (see both Figure 6a and Figure 7). The process of recording the load profile of the filled wagon 12a then comprises processing the 3D images to generate a 3D composite image 119 (see Figure 6a), as will be described in more detail below.

The composite image 119 is preferably expressed in terms of a track coordinate system. The track coordinate system uses a local (right-hand) Cartesian coordinate system (x, y, and z) to define the wagon profile with the axes defined as follows: • X Axis: a horizontal axis aligned with the track rails, with the positive direction towards the rear of the wagon 12;

• Y Axis: a horizontal axis perpendicular to, and in an anti-clockwise direction from, the positive x-axis (right-hand coordinate system);

· Z Axis: a vertical axis perpendicular to the x- and y- axes, with the

positive direction upwards towards the cameras 14.

The wagon profile X and Y axes are defined with reference to the track reference position, located outside the leading corner of the wagon 12. The wagon profile Z axis is defined with reference to the track reference level. Normally, the position and orientation of each camera 14 is fixed in relation to the track coordinate origin. Preferably the position and orientation of each camera 14 is accurately measured at step 102 to define the optical axis of each camera in relation to the track coordinate system (track X, Y and Z axes). Target position data supplied by each camera 14 is preferably mapped from a camera coordinate system to the track coordinate system.

The 3D time-of-flight cameras 14 return a target distance for each pixel in the field of view (FOV). For a camera with a pixel array size of 320 (h) x 120 (v), there will be 38,400 target distance values returned in each frame. The angular resolution depends on the camera FOV. For a FOV of 40° (h) x 30° (v), the angular resolution will be 0.125° (h) x 0.250° v). The position of the target point for each pixel is defined in term of the camera coordinate system (see Figures 4 and 5).

The depth distance (Z) produced by the camera is the perpendicular distance from the target point to the lens entrance pupil plane (the entrance pupil plane is behind the front glass of the camera). The depth distance is different from the range distance which is the straight line distance from the target point to the corresponding pixel in the lens entrance pupil plane (see Figure 4). Note that for the target point lying on the optical axis of the camera, the depth and range distances are the same. The camera coordinate reference point (x=0, y=0 and z=0) is located where the optical axis intersects the lens entrance pupil plane. The position of each target point is described by the target distance along the z axis and the angular offset along the camera x and y axis. Advantageously each camera 14 is capable of providing target point data at a high frame rate (typically up to 60 frames per second), with an image capture time of less than ten milliseconds. Preferably the high frame rate provides the ability to present a continuously updated wagon profile shown at step 106 in Figure 6 and at step 116 in Figure 7. The fast image capture also provides for accurate capture whilst the wagon is moving.

In creating a wagon profile from the camera target values, it is important to:

• Preserve the accuracy of the camera target distance measurement.

• Store the wagon profile data in a format that facilitates accurate

determination of the wagon edges and volume.

· Maintain data storage space requirements in a wagon profile database 32 within manageable boundaries.

Preferably the target points from all cameras 14 are mapped into a wagon profile point map in order to provide a plan view profile of the wagon 12a. Typically the wagon profile is a two dimensional height map array 40, as shown in Figure 3, one dimension of the array extending parallel to the rail direction whilst the second dimension extends perpendicular to the rail direction. The number of elements in each dimension is typically selected to match the available resolution of the cameras 14. The height value in each array element represents the third dimension and is expressed as a height above the track coordinate reference level.

Mapping the camera target points (expressed in terms of the camera coordinates) to the track coordinate system is accomplished by transformation (rotation and translation) of the target points via a camera to track Transform Matrix, shown at steps 108 and 110 in Figures 6a and 7. The camera position is described by the translation of the camera coordinate reference point (where the optical axis intersects the lens entrance pupil plane) with respect to the track coordinate system reference point. Thus for a camera 14 mounted 7m above the centre of the track 16, and 2m from the track datum along the x axis, the translation is x=2.0, y=0.0 and z=7.0. The camera orientation is described by the direction (rotation) of the optical axis (2 axis) with reference to the track x axis and the direction (rotation) of the camera y axis with reference to the track y axis. The camera orientation is expressed as a Quaternion but can also be expressed as Euler Angles or a Rotation Matrix. The orientation quaternion and position translation are combined to provide the Camera Transform Matrix at step 108.

The height maps for each of 3D cameras 14 are merged at step 112 to form a composite image 119. Any image points of structural elements of the train or ore loading facility (delivery chute, etc), which encroach on the cameras' field of view are culled at step 114 to produce a complete panoramic three dimensional picture of the wagon profile at step 106 (see Figure 6b). Camera target points corresponding to structural elements are ignored when mapping the target data to the wagon profile array. This is accomplished by provision of 3D models of the structural elements at step 118, accurate measurement of the location and orientation of the structural elements at step 120, and transformation of the 3D models of the structural elements at steps 122 and 124. Target points falling within the 3D model space are ignored. Culling for a single structure is illustrated. Additional structures may be added.

Maintaining camera alignment and calibration is essential for accurate profiling. An alignment and calibration method is described that provides automated alignment of each camera in relation to the track coordinate system and to verify correct target distance accuracy.

Automatic camera alignment is facilitated by the accurate measurement of the rails during installation and by the installation of calibration tags. The track rails 16 are utilised for axis direction alignment. The alignment of each camera position along the x axis is facilitated by the installation of x axis calibration tags. Each tag defines a calibrated position along the x axis. Each x axis calibration tag may be implemented as a short raised area between the rails. The length of the raised section along the x axis should be selected such that the section edge is detectable. The raised section may extend from rail to rail along the y axis. For each camera 14, alignment is performed by determining the camera target points corresponding to the rails, and then adjusting the camera to track transform matrix to align the points along the rail axis. Camera points corresponding to the rails are determined by firstly only considering points that are in a pre-defined rail zone and then comparing the heights of adjacent target points to determine the points corresponding to a line running along the top of the rail. The rail point set is then processed with a best fit algorithm to determine a directional vector for the point set. The camera to track transform matrix is calculated to align the rail target point set with the known rail directional vector. The measured rail directional vector is compared to a vector based on the installed camera position to verify that the camera is within a reasonable alignment range.

When the wagon profile has been determined at step 106 in Figure 6b, the profile is stored in the database 32. It can be combined with a wagon model to calculate or detect various parameters of the wagon load. The wagon is identified in the database 32 from an externally provided wagon identifier, or alternatively by a train identifier and wagon number counted as the wagons pass the profiling station. Advantageously the wagon profile obtained prior to loading can be used to identify a wagon type, dimensions and capacity.

The edges of the wagon are determined at 132 from the scanned profile based on the sharp transition in profile level.

Predetermined wagon identifiers 126 are used to select an appropriate wagon model from a suite of wagon models 128 at step 130. This information relating to the wagon model can be combined with the wagon profile obtained at 106 to calculate the loaded wagon volume at step 134. The wagon model information can also be combined with the wagon profile and edge position data to detect wagon overload (step 136) as well as an unbalanced wagon load (step 138). All this information can be simultaneously displayed on monitor 30 and stored in database 32.

The high frame rate (typically up to 60 frames per second), and image capture time of less than ten milliseconds, of the 3D time-of-flight cameras 14 not only allows a continuously updated visualisation of the wagon profile, it may also allow direct control of the loading process, as well as:

• Determination of spillage

• Detection of material hang-up during or after unloading

• Detection of personnel or material in unloaded wagons

· Identification of wagon type from dimensions

Figure 7 illustrates the process involved in controlling the loading of an empty wagon 12b. The process of obtaining a merged 3D image 112 of the wagon profile is similar to that described above for recording the 3D wagon profile 106. In order to monitor the filling of the wagon 12b, the wagon model image points are culled at step 116 in Figure 7 to calculate the dynamically varying load profile at step 121. At the same time profile visualisation is provided (step 104) on monitor 30. In order to determine the correct wagon model, the wagon edges are determined at step 132. This is performed in a similar manner to that described above in relation to Figure 6b. The information relating to the wagon type is also employed to obtain the appropriate loading control parameters at step 123, which determines the desired fill volume profile 125 and fill volume rate curve 127.

Once the dynamically varying load profile has been calculated (step 121), load visualisation can be provided (step 129) on monitor 30, the wagon load volume can be calculated (step 131) and wagon overload detection implemented (step 133). Wagon overload detection is performed at 133 using the dynamically varying load profile 121 together with the wagon model information. Likewise, determination of the wagon load volume at step 131 is performed using the dynamically varying load profile 121 together with the wagon model information. The dynamically varying wagon load volume is used to calculate the actual fill volume rate 142 at step 140.

The actual fill volume rate 142 is supplied to the gate controller for controlling the gate position of the discharge hopper 144 (see Figure 2), and compared with the desired fill volume rate curve 127 to control the rate at which the wagon 12b is filled (see Figure 8).

Now that preferred embodiments of the 3D wagon profiler method and apparatus have been described in detail, it will be apparent that the described embodiments provide a number of advantages over the prior art, including the following:

(i) Providing accurate measurement of the wagon dimensions and capacity prior to loading; and, determining the type of equipment to discriminate between valid ore wagons and rail equipment such as locomotives, compressor brake cars and track maintenance vehicles; and, determining the presence of foreign objects, material or water in ore wagons which could lead to product contamination.

(ii) Detecting wagon misalignment based on the relative level of the wagon edges, before, during and after loading. (iii) Controlling the wagon fill rate to avoid spillage during the loading process.

(iv) Accurate 3D profiling of wagons over the full length of the.wagon.

(v) Reliable determination of correct wagon loading, including the load profile just inside the leading and trailing edges of a wagon. (vi) The 3D cameras are able to capture the wagon profile at high frame rates to provide a continuously updating the wagon and load profiles. (vii) Measuring and recording the wagon loaded profile for evidence of the dispatched train load; and, indicating and recording the occurrences of wagon under or over loading.

(viii) Measuring the wagon loaded profile at the destination and evaluating the volume change.

(ix) Determining the type of equipment at the entrance and exit of a wagon dumping station to identify rail equipment that cannot enter or be dumped.

(x) Facilitating the implementation of autonomous train loading control by overcoming the shortcoming of prior art wagon loaders in controlling the loading process.

It will be readily apparent to persons skilled in the relevant arts that various modifications and improvements may be made to the foregoing embodiments, in addition to those already described, without departing from the basic inventive concepts of the present invention. For example, in the described embodiments the 3D time-of-flight cameras provide certain advantages as noted above; however it will be understood that other suitable types of 3D image sensor may also be employed. Therefore, it will be appreciated that the scope of the invention is not limited to the specific embodiments described.