BILLET IN A STACK OF BILLETS

FIELD OF THE INVENTION

[0001] The present invention relates to a method and a system for locating a billet in a stack of billets using imaging.

BACKGROUND OF THE INVENTION

[0002] In the metal production industry, the tracking and the locating of the metal product (for example, a billet) is important. Reliable locating of a billet, for example, would allow for a more efficient manufacturing process flow.

[0003] The known tracking methods rely on the embedding of, for example, tags, or the marking of the metal product followed by vision-based recognition. However, metal product making (for example, continuous casting) is a high temperature process. Thus, the automatic and reliable locating of the billet remains a challenge.

[0004] To solve this problem, vision-based automatic tracking methods have been proposed

[0005] CN 106780483, for example, discloses a method for obtaining billet characteristics and extracting the coordinates of the center of the end surface of the billet through image processing in a continuous casting billet production environment to build an automatic positioning system for automatically identifying the cast billet to improve the labeling efficiency and labeling accuracy of the continuous casting process. [0006] Vision-based Automatic Identification Tracking of Steel Products for Intelligent Steel Manufacturing (Chao- Yung Hsu, 2017 IEEE International Symposium on Multimedia, pp. 376-377), for example, discloses a method for automatically tracking steel products using a vision-based automatic identification tracking framework (without embedding any identification codes) by extracting visual features from the steel products.

[0007] Depth-based Feature Extraction-Guided Automatic Identification Tracking of Steel Products for Smart Manufacturing in Steel 4.0 (Chao-Yung Hsu, Proceedings of IEEE International Conference on Applied System Innovation 2018, pp. 145-146), for example, discloses a method that uses vision-based automatic identification tracking (without embedding any identification codes onto the product surface), which method also includes extracting visual features from the steel image.

SUMMARY OF THE INVENTION

[0008] It is an object of the present invention to provide a locating method and a locator system to locate a billet in a stack of billets without using a tag or a marker.

[0009] In continuous casting, a billet may be cut out of the cast material using a flame cutter. The flame cutter leaves a leading end surface and a trailing end surface each having a unique surface pattern due to the flame cutting.

[0010] In a locating method and a locator system according to the present invention the surface pattern of the leading end surface and/or the surface pattern of the trailing end surface of a billet, that has been cut using a flame cutter, is used to identify the billet.

[0011] A locator system according to the present invention is configured to locate a flame-cut billet in a stack of flame-cut billets, and to send location information of a located billet to, for example, a billet marker, a billet transporter, a billet location storage system, or a billet location display system. [0012] The locator system may include an imager having at least one digital camera (preferably two digital cameras) that can acquire a digital image of a first end or a second end of a billet in a stack of billets, an image classifier that includes at least one computer configured to classify a surface having surface characteristics resulting from flame cutting, and configured to classify a digital image of a first end or a second end of a billet that is received from the imager, and an image matcher that includes a computer configured to match a classified digital image of a first end or a second end of a billet that is received from the image classifier with a classified image from among a plurality of classified images stored in a classified image storage.

[0013] The image classifier preferably classifies the digital image of the first end or the second end of the billet using pattern classification and recognition, which is a known method of classification and recognition of surface characteristics of an object.

[0014] The image matcher may match the classified digital image of the first end or the second end of the billet (target image) by comparing the target image to the plurality of classified images stored in the classified image storage, and, if the comparing does not result in a match, by further comparing an angularly rotated target image to the plurality of classified images stored in the classified image storage until a match is made, or no match is made.

[0015] In one embodiment, the image matcher may rotate the target image to obtain the angularly rotated target image.

[0016] In another embodiment, the image matcher may request the image classifier to angularly rotate the target image, and performs the matching based on an angularly rotated target image received from the image classifier. [0017] The classified image storage may be connected to another image classifier that classifies a digital image of a leading end or a trailing end of a billet captured by a digital camera (preferably two digital cameras) of another imager after the billet is cut with a flame cutter. The at least one digital camera (or the two digital cameras) are capable of capturing images of the leading end or the trailing end of the billet at a temperature of up to 1200°C.

[0018] The another image classifier also classifies a surface having surface characteristics resulting from flame cutting, using, preferably, pattern classification and recognition.

[0019] The another image classifier, for each billet, classifies an image of a leading end of the billet and an image of a trailing end of the billet and may store the classified leading and trailing end images in the classified image storage in association with one another.

[0020] The image matcher may first match the classified digital image of the first end or the classified digital image of the second end (target image) of the billet to either a classified leading end image or a classified trailing end image (reference image) stored in the classified image storage, and, after the target image is matched with a reference image, the image matcher may match a classified image associated with the matched reference image with the classified digital image of the end of the billet that is opposite the target image.

[0021] The image matcher may match the classified digital image of the target image by comparing the target image to the plurality of classified images stored in the classified image storage, and, if the comparing does not result in a match, by further comparing an angularly rotated target image to the classified images stored in the classified image storage until a match is made, or no match is made. [0022] In one embodiment, the image matcher may rotate the classified digital image of the first end or the second end of the billet to obtain the angularly rotated target image.

[0023] In another embodiment, the image matcher may request the image classifier to angularly rotate the image of the target image, and performs the matching based on the angularly rotated target image received from the image classifier.

[0024] A locating method according to the present invention locates a flame-cut billet in a stack of flame-cut billets by capturing an image of an end of a billet in a stack of billets with an imager that includes at least one digital camera that can acquire a digital image, classifying the image of the end of the billet captured by the imager using an image classifier that includes at least one computer configured to classify a surface having surface characteristics resulting from flame cutting, and matching the classified image of the end of the billet to a classified image from among a plurality of classified images in a classified image storage using an image matcher that includes a computer configured to match a classified digital image of an end of a billet with a classified image from among a plurality of classified images stored in the classified image storage.

[0025] The classifying may be performed using pattern classification and recognition.

[0026] The matching may be performed by comparing the classified image of the end of the billet (target image) to the plurality of classified images stored in the classified image storage, and, if the comparing does not result in a match, by further comparing an angularly rotated target image to the plurality of classified images stored in the classified image storage until a match is made, or no match is made. [0027] In one embodiment, the angularly rotated target image can be obtained by rotating the target image in the image matcher, or, in another embodiment, rotating the target image with the image classifier.

[0028] The matching may be performed by first matching the target image to either a classified leading end image or a classified trailing end image (reference image) stored in the classified image storage, and, after the target image is matched with a reference image, matching a classified image associated with the matched reference image with the classified digital image of the end of the billet that is opposite the target image.

[0029] Other features and advantages of the present invention will become apparent from the following description of the invention which refers to the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGiS)

[0030] Fig. 1 illustrates a system according to the present invention.

[0031] Fig. 2 illustrates an embodiment of the present invention in a deployed state.

[0032] Fig. 3 illustrates an example of an input side of a system according to the present invention.

[0033] Fig. 4 illustrates an example of an output side of a system according to the present invention.

[0034] Fig. 5 illustrates an image of an end of a billet captured at the input side of the system.

[0035] Figs. 6A-6D illustrate images of an end of a billet captured at the output side of a system according to the present invention in different orientations. [0036] Fig. 7 is a flow diagram of the steps in a method carried out by a system according to the present invention.

DETAILED DESCRIPTION OF EMBODIMENTS OF THE INVENTION

[0037] A “billet” or “flame-cut billet” as used herein refers to a billet cut from a continuously cast metal product using a flame cutter. An “end” of the billet refers to the surface at the end between the top, the bottom, and the side edges which exhibits characteristic features resulting from flame cutting.

[0038] Referring to Fig. 1, a method according to the present invention is implemented with a system 10 that includes a first imager 12, a first image classifier 14, a classified image storage 16, a second imager 18, a second image classifier 20, and an image matcher 22.

[0039] The first imager 12 is an image acquisition system that includes at least one camera, preferably a digital camera, that is suitable for acquiring a digital image of a leading end or a trailing end of a billet while the billet is hot. The exterior surface temperature of the billet may be up to 1200°C. The exterior surface temperature of the billet may be as low as the ambient temperature (for example, 25°C). The exterior surface temperature of the billet depends on the speed of casting, the slower the casting speed the lower the temperature. The first imager 12 may include two digital cameras, one provided to acquire a digital image of a leading end of the billet, and the other provided to acquire a digital image of the trailing end of a billet.

[0040] The first image classifier 14 is a system that includes a computer or a plurality of computers operatively connected to the first imager 12 to receive and classify digital images of the leading ends and the trailing ends of the billets from the first imager 12. Thus, in operation, the first image classifier 14 will classify the leading end image and the trailing end image of a billet acquired by the first imager 12 [0041] The classified image storage 16 is a system that includes a computer or a plurality of computers configured (programmed) to receive and retain computer- readable files in computer-readable, non-volatile memory locations, the computer- readable files containing information pertaining to each classified (leading or trailing end) image that is classified by the first image classifier 16.

[0042] The second imager 18 is an image acquisition system that includes at least one camera (preferably a digital camera) that is suitable for acquiring a digital image of a first end or a second end of a billet. The second imager 18 may include two digital cameras, one for acquiring an image of the first end of a billet, and another for acquiring an image of the second end of the same billet.

[0043] The second image classifier 20 is a system that includes a computer or a plurality of computers operatively connected to the second imager 18 to receive, and configured (programmed) to classify digital images of the first end and the second end of a billet that are received from the second imager 18.

[0044] The image matcher 22 is a system that includes a computer or a plurality of computers operatively connected to the second image classifier 20 to receive computer-readable files containing information pertaining to each classified image of an end of the billet that is classified by the second image classifier 20.

[0045] The image matcher 22 is configured (programmed) to communicate with the classified image storage 16 and to match an image of the first end of the billet and an image of the second end of the billet classified by the second image classifier 14 with an image of the leading end of the billet or the trailing end of the billet classified by the first image classifier 14 that is stored in the classified image storage 16. The image matcher 22 is in communication with a user interface 23 (for example, a computer that includes also a visual display and an input device, such as a keyboard) for receiving instructions to match an image. [0046] Following the image matching by the image matcher 22, the system 10 may take steps such as mark the billet (e.g. with a stamp or a paint spray nozzle), identify and store (in a computer memory) the location of the billet in the stack of billets, or operate a forklift, a crane, or similar means, to move the identified billet from its location to another location.

[0047] The first image classifier 14 and the second image classifier 20 are both programmed to perform a method to create a computer-readable file containing information that uniquely identifies each image of an end of a billet that is acquired by an imager. The classification method may be based on pattern classification and recognition, which can classify and recognize a surface pattern.

[0048] The classified image storage 16 stores at least the computer-readable files produced by the first image classifier 12 in a computer-searchable manner. Preferably, the classified image storage 16 includes a document management system or the like system.

[0049] Referring now to Fig. 2, a system 10 according to the present invention includes an input side 24 and an output side 26.

[0050] The input side 24 includes the first imager 12, the first image classifier 14, and the classified image storage 16.

[0051] The output side 26 includes the second imager 18, the second image classifier 20, and the image matcher 22.

[0052] The input side 24 may be arranged at a billet production facility 28 that, for example, produces billets through a continuous casting process.

[0053] The output side 26 may be arranged at a metal storage facility 30 that, for example, stores billets for production of metal products, such as sheet metal products, beams or pipes. The metal storage facility 30 may be part of a metal product production facility, or may be a facility used solely for the purpose of storing billets produced at the billet production facility 28.

[0054] Referring to Fig. 3, at the billet production facility 28, a first digital camera 32 and a second digital camera 34 (which are part of the first imager 12), are installed at a location to capture an image of the leading end and the trailing end of a billet after the billet is cut from the continuous cast using, for example, a flame cutter. Specifically, for example, the first camera 32 is arranged to capture an image of the leading end 35 of the billet 36, and the second camera 34 is arranged to capture an image of the trailing end 37 of the billet 36 after the billet 36 after the billet is cut using a flame cutter. Preferably, the leading end 35 image and the trailing end 37 image of the billet are captured by cameras 32, 34 while the billet is still hot, i.e. as hot as 1200°C.

[0055] The image of the leading end 35 of the billet 36 that is captured by the first camera 32, and the image of the trailing end 37 of the billet 36 that is captured by the second camera 34 are sent to the first image classifier 14. The first image classifier 14 is programmed to classify the image of the leading end 35 of the billet 36 and the trailing end 37 of the billet 36 using pattern classification and recognition. Once each image is classified, both end images are stored in the classified image storage 16 and associated with one another as a pair to indicate that the images are the leading end image and the trailing end image of the same billet.

[0056] The billet 36 is then transported to the metal storage facility 30, which may be a location remote from the billet production facility 28.

[0057] The output side 26, which may be arranged at the metal storage facility 30, includes a third camera 38, and a fourth camera 40 (which are part of the second imager 18). Cameras 38 and 40 are preferably digital cameras. A stack 42 of billets is stored at the facility 30, and billet 36 would be stored in the stack among other billets. [0058] In order to identify the location of each billet, including billet 36, each camera 38, 40 is moved automatically to capture an image of an end of a billet in the stack. That is, the third camera 38 captures an image of a first end 39 of a billet and the fourth camera 40 captures an image of the second end 41 of the same billet.

[0059] The images of the first end 39 and the second end 41 so captured by the third and the fourth cameras 38, 40 are fed to and received by the second image classifier 20. The second image classifier 20 is programmed to classify the end images received from the cameras 38, 40 using pattern classification and recognition. The images so classified are then sent to the image matcher 22.

[0060] The image matcher 22 then takes the classified image of the first 39 and the second 41 end of the billet and proceeds to match each classified image to a classified image of a leading end or a trailing end of a billet stored in the classified image storage 16. For example, the image matcher 22 may first try to match a classified image received from the second image classifier 20 to a classified leading end image. If that does not result in a match, the image matcher 22 proceeds to match the classified image received from the second image classifier 20 to a classified trailing end image that is stored in the classified image storage 16.

[0061] To match a classified image received from the second image classifier 20 (hereafter referred to as the “target image”) to a classified leading end image or a classified trailing end image, the target image may need to be rotated because, for example, the orientation of the billet 36 in the stack 42 may not be the same as its orientation when images of the leading end 35 and its trailing end 37 were captured by the first imager 12.

[0062] Specifically, referring to Fig. 5, which illustrates an end of the billet 36 that could be a leading end 35 or a trailing end 37, when the first imager 12 captures an image of the end 35 or 37 using a camera 32 or 34, the image of the end will have a top edge 44, a bottom edge 46, a right side edge 48, and a left side edge 50. The image of the end 35 or 37 will be classified by the first image classifier 14 relative to the edges 44, 46, 48, 50 and their respective relative positions to one another.

[0063] At the output side 30, when placed in the stack 42, the billet 36 may be rotated such that, for example, the top edge 44 will be located at the bottom. Thus, the image of an end 35 or 37 captured by the second imager 18 will be rotated one hundred eighty degrees relative to the image of the end 35 or 37 taken by the first imager 12. The image of an end 39 or 41 needs to be reoriented every time the matcher 22 makes a comparison of a target image to a classified leading end image or a classified trailing end image in the classified image storage 16.

[0064] Fig. 6A illustrates an image of an end 39 or 41 captured by a camera 38 or 40 of the second imager 18, which will be used as the target image. The target image so captured will also have a top edge 52, a bottom edge 54, a right side edge 56, and a left side edge 58. However, edges 52, 54, 56 and 58 may not have the same relative positions as the edges 44, 46, 48 and 50 of the image of an end 32 or 37 taken by the first imager 12. Thus, when matching, if no match is made between the target image and a classified leading edge or a classified trailing edge, the target image is rotated 90 degrees (see Fig. 6B), clockwise or counter-clockwise, by the image matcher 22 and the rotated target image is compared to the classified leading end images (or the classified trailing end images) and then, if no match is obtained, the classified trailing end images (or the classified leading end images) stored in the classified image storage 16, until the target image is matched with a classified leading end image or a classified trailing end image. Once the image is matched with a leading end image or a trailing end image (reference image), the classified second end image becomes the target image and is matched to the other image that is paired/associated with the classified image that was matched with the first target image. Thus, for example, if the first target image (classified image of the first end 39) is matched with a classified leading end image, the target image (the classified end 41 image) is matched with the classified trailing end image that is associated with the first matched target image to verify the match. Once the image matcher 22 makes the match, the location of the billet in the stack 42 is stored for future retrieval, or the identified billet is marked automatically.

[0065] Referring to Fig. 7, a method 100 according to the present invention includes capturing an image of the leading end of a billet 36 by the camera 32 (S101), capturing an image of the trailing end 37 of the billet 36 by the camera 34 (SI 02), receiving and classifying the image captured at S101 with the first image classifier 14 (S103), storing the classified image classified at S103 in the classified image storage 16 (SI 04), receiving and classifying the image captured at SI 02 by the first image classifier 14 (S105), storing the classified image that is classified at S105 in the classified image storage 16 (SI 06), associating as a related pair the classified images stored in steps SI 04 and SI 06 (SI 07), receiving instruction at the output side 26 from the user interface 23 a request to locate a billet (for example billet 36) in the stack 42 (S108), capturing an image of the first end 39 of a billet in the stack 42 with the third camera 38 (SI 09), capturing an image of the second end 41 of the same billet with the fourth camera 40 (SI 10), receiving and classifying the first end 39 image captured at SI 09 with the second image classifier 20 (SI 11), receiving and classifying the second end 41 image captured at SI 10 with the second image classifier 20 (SI 12), optionally, storing the classified images generated in steps Sill and SI 12 in a computer memory located at the facility 30 (SI 13), comparing the classified image classified at Sill (first target image) with all classified leading end images stored in the classified image storage 16 to find a match (SI 14), if no match is found in step SI 14, rotating by 90° (90 degrees) the first image that was compared in step SI 14 and repeating the comparing step SI 14 (SI 15), if no match is made in step SI 15, rotating by 180° (one hundred eighty degrees) the first image that was compared in step SI 14 and repeating the comparing step SI 14 (SI 16), if no match is made in step SI 16, rotating by 270° (two hundred seventy degrees) the first target image that was compared in step SI 14 and repeating the comparing step SI 14 (SI 17), if step SI 17 does not result in a match, repeating steps SI 14-SI 17 but, instead of comparing the first target image to the classified leading images, comparing the first target image with the trailing end images (SI 18), if no match is made in step SI 18, issuing an error generating and sending a message to the user interface 23 indicating that no match was made (SI 19), if a match is found in any one of the steps S114 to S118, comparing the classified image classified at SI 12 (the second target image) with the classified end image stored in the classified image storage 16 in association with the classified end image that was matched with the first target image to confirm a match (S120), if no match is found in step S120, rotating by 90° (90 degrees) the second target image that was compared in step SI 20 and repeating the comparing step S120 (S 121), if no match is made in step S121, rotating by 180° (one hundred eighty degrees) the second target image that was compared in step S120 and repeating the comparing step S120 (S122), if no match is made in step S122, rotating by 270° (two hundred seventy degrees) the target image that was compared in step S120 and repeating the comparing step S120 (S123). If steps S120 to S123 do not result in a match, the method may include issuing an error message to the user interface device 23 indicating that no match was found (S124).

If a match is found in any one of the steps S120, S121, S122, S123, the method may include identifying and recording in a computer memory located at the facility 30 the location of the matched billet as billet 36 in the stack 42 for future retrieval or marking of the billet 36 (S125).

[0066] In another embodiment, the matcher 122 would not rotate a classified image. The matcher 22 may send a request to the second image classifier 20 for a classified, rotated image. Upon receipt of the request, the second image classifier 20 will rotate the image (by first 90 degrees, and thenl80 degrees, and then 270 degrees as deemed necessary), classify the rotated image, and send the classified rotated image to the matcher 22 to use as a target image for comparison with classified leading end images or the classified trailing end images as disclosed above.

[0067] Rotating an image or a classified image refers to virtually rotating the image, which may or may not be displayed by a computer monitor. To rotate an image or a classified image, the coordinates are angularly shifted around a central point axis, clockwise or counter-clockwise. The coordinates of the central axis would not be angularly shifted. Thus, the rotation would take place relative to a coordinate that remains invariable.

[0068] The method (system) disclosed herein should not be understood to be restricted to the identification of a billet in a stack, but can be applied to any group or collection of billets. Thus, the word “stack” as used herein is not limited to an arrangement in which one billet is placed on another billet, but to any grouping or collection of billets.

[0069] Although the present invention has been described in relation to particular embodiments thereof, many other variations and modifications and other uses will become apparent to those skilled in the art. It is preferred, therefore, that the present invention be limited not by the specific disclosure herein, but only by the appended claims.