Description METHOD FOR PROCESSING THREE-DIMENSIONAL IMAGE

AND COMPUTER READABLE MEDIUM STORING COMPUTER PROGRAM FOR PERFERMING THE METHOD

Technical Field

[1] The present invention relates to a method for processing a three-dimensional image, and in particular, to a specific three-dimensional image processing method for obtaining a single three-dimensional object image directly through a merging and correcting process over a raw (preprocessed) set of multiply divided images transmitted from a three-dimensional scanning apparatus, and to a readable storage medium equipped with a program capable of performing the same. Background Art

[2] In general, three-dimensional scanning of an object is obtained by projecting optical pattern images on the object and then capturing digitally the same projected images, using a three-dimensional scanning apparatus composed of a three-dimensional scanner and a three-dimensional digitizer.

[3] Techniques for scanning an object include an optical triangulation method which projects laser beam to an object and analyzes the reflected beam to obtain a three- dimensional image of the object, and a Moire method which places an object in a grid having stripes of a regular interval such as wavy patterns and makes a three-dimensional shape of the object using an interference phenomenon that the stripes are seen as curved lines along the object.

[4] The optical triangulation method is performed such that laser beam is projected to an object from a laser diode and the reflected laser light deformed along a surface curve of the object is obtained by a CCD camera, and thus three-dimensional coordinates of the object are calculated from a geometric relationship. Here, the optical triangulation method measures the object by scanning while vertically moving using a machine driving system.

[5] The Moire method is performed such that a plurality of pattern films having horizontal and vertical lines operated by a rotary wheel which rotates subsequently are rotated subsequently, a plurality of pattern images projected by a lamp form horizontal or vertical shades on surfaces of an object, the shades are deformed according to the shape of the object to form a wavy contour having a shape information of the object, and the shape information is analyzed to obtain three-dimensional coordinates.

[6] The above-mentioned three-dimensional scanning apparatus transmits captured images scanned using the optical triangulation method or the Moire method to a

computer. At this time, the three-dimensional scanning apparatus divides the object by various angles and scans the object so as to show a three-dimensional image of the object on the computer in the whole. That is, takes a Venus plaster figure as an example, the apparatus divides a Venus plaster figure of the object into multiply areas from at least front and rear, to front, rear, left, right, top and bottom, and scans the figure.

[7] Here, a plurality of captured images transmitted to the computer are merged and corrected to process three-dimensional images, and a conventional three-dimensional image processing step is described below with reference to FIG. 1.

[8] As shown in FIG. 1, an operator inputs each order for extracting a first captured image to an Nth captured image stored in a database of a computer to extract a plurality of captured images of an object (Sl), repositions and merges the extracted captured images on a single spatial coordinate system by a manual operation, such as mouse drag (S2), and corrects the merged image by a manual operation using mouse click or a touchpen to generate a plurality of captured images to three-dimensional images (S3 and S4).

[9] However, the conventional three-dimensional image processing step depends on a manual operation of an operator over the whole steps, including captured image extraction, the extracted image merging, and the merged image correction, and thus requires much time. And, the conventional three-dimensional image processing step has a limitation in precisely merging and correcting images according to conditions of the operator, and a problem of an increase of a faulty image generation rate caused by a manual operation. Disclosure of Invention

Technical Problem

[10] The present invention is designed to solve the above-mentioned problems, and therefore one of the purposes of the present invention is to provide a method for processing a three-dimensional image which automatically converts a plurality of captured images transmitted from a three-dimensional scanning apparatus into a single three-dimensional object image by a computer, and a computer readable storage medium storing a computer program for performing the method.

[11] And, another purpose of the present invention is to provide a user with improved working conditions by allowing the user to easily obtain three-dimensional images of an object. Technical Solution

[12] In order to achieve the above-mentioned objectives, in a method which processes a captured image of an object scanned by a three-dimensional scanning apparatus to a

three-dimensional image, a method for processing a three-dimensional image according to the present invention includes (A) the three-dimension scanning apparatus dividing the object into at least two areas, scanning the object and transmitting a captured image for each area of the scanned object to a host computer; (B) converting the captured image for each area into three-dimensional preprocessed image data; and (C) merging the three-dimensional preprocessed image data to form a single three- dimensional image data.

[13] According to the present invention, the step (A) includes (Al) the three-dimensional scanning apparatus projecting light of a predetermined wavelength range to the object by each area and capturing an image generated on a surface of each area of the object; (A2) converting the captured image for each area of the object into a digital signal; and (A3) transmitting the digital signal to the host computer.

[14] Preferably, the object includes a human body, an animal or a thing, the three- dimensional scanning apparatus of the step (Al) has an optical projector and a sensor camera that move around the object, and the light of the step (Al) is a laser beam or a near- infrared ray.

[15] In the present invention, the step (B) includes (Bl) extracting three-dimensional coordinate values at intersections of the captured image for each area and a reference coordinate system; (B2) converting the three-dimensional coordinate values into a point cloud; and (B3) converting the point cloud into three-dimensional data (STL; Standard Tesselation Language).

[16] Preferably, the reference coordinate system of the step (B 1) is a three-dimensional fixed grid system of which a reduced scale is matched with a setting space where the object is set in the three-dimensional scanning apparatus.

[17] According to the present invention, the step (C) includes (Cl) a registering step for arranging the three-dimensional preprocessed image data for each area of the object in a coordinates system matched with a reference coordinate system; (C2) a merging step for merging the registered three-dimensional preprocessed image data to form a single three-dimensional image data; (C3) a decimation step for decimating the merged three- dimensional image data; and (C4) a hole filling step for correcting blind spots and indented areas of the decimated three-dimensional image data.

[18] Preferably, in the step (C2), an overlapping area of the three-dimensional pre- processed image data is removed to generate an interface area of which continuity is ensured.

[19] And, in the step (C3), the number of pixels of the three-dimensional image data is reduced while maintaining the original shape of the object.

[20] Furthermore, in the step (C4), the blind spots and indented areas are corrected by decreasing or increasing the number of pixel under the estimation of three-dimensional

coordinate values of adjoining data.

[21] According to the present invention, the steps (A) to (C) are automatically performed by the host computer.

[22] According to another aspect of the present invention, a computer readable storage medium is provided for storing a computer program for performing the method for processing a three-dimensional image. Brief Description of the Drawings

[23] Preferred embodiments of the present invention will be more fully described in the following detailed description, taken accompanying drawings. However, it should be understood that the description proposed herein is just a preferable example for the purpose of illustrations only, not intended to limit the scope of the invention. In the drawings:

[24] FIG. 1 is a flowchart of a conventional three-dimensional image processing step.

[25] FIG. 2 is a block diagram illustrating arrangement of a three-dimensional image processing system in accordance with the present invention.

[26] FIG. 3 is a flowchart of a step for transmitting a captured image of an object to a host computer in accordance with the present invention.

[27] FIG. 4 is a view illustrating multiply division scanning of an object in accordance with a preferred embodiment of the present invention.

[28] FIGs. 5 and 6 are views illustrating a step for projecting optical pattern images to an object in accordance with preferred embodiments of the present invention.

[29] FIG. 7 is a flowchart of a method for processing a three-dimensional image in accordance with a preferred embodiment of the present invention.

[30] FIGs. 8 to 10 are views illustrating steps for converting a captured image into three- dimensional data by a three-dimensional image preprocessing step in accordance with a preferred embodiment of the present invention.

[31] FIG. 11 is a view illustrating a step for processing the captured image to three- dimensional data in accordance with another preferred embodiment of the present invention.

[32] FIGs. 12 and 13 are views illustrating steps for merging a three-dimensional image in accordance with a preferred embodiment of the present invention. Best Mode for Carrying Out the Invention

[33] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. Prior to the description, it should be understood that the terms used in the specification and the appended claims should not be construed as limited to general and dictionary meanings, but interpreted based on the meanings and concepts corresponding to technical aspects of the present

invention on the basis of the principle that the inventor is allowed to define terms appropriately for the best explanation. Therefore, the description proposed herein is just a preferable example for the purpose of illustrations only, not intended to limit the scope of the invention, so it should be understood that other equivalents and modifications could be made thereto without departing from the spirit and scope of the invention.

[34] According to the present invention, three-dimensional image processing is performed such that a captured image of an object transmitted from a three-dimensional scanning apparatus is automatically converted into three-dimensional image data by a three-dimensional image processing program included in a host computer. At this time, the object includes a human body, an animal or a thing.

[35] A three-dimensional scanning apparatus and a host computer for three-dimensional image processing are described below with reference to FIG. 2.

[36] As shown in FIG. 2, the three-dimensional scanning apparatus 100 includes a scanning unit 110, a control unit 120 and a communication interface 130, and the host computer 200 includes an input unit 210, an output unit 220, a database 230, a communication interface 240, a BUS 250 and a central processing unit (CPU) 260.

[37] In the three-dimensional scanning apparatus 100, the scanning unit 110 projects pattern images to an object and captures the images, the control unit 120 controls the scanning unit 110 based on data transmitted from the host computer 200, the communication interface 130 exchanges data between the three-dimensional scanning apparatus 100 and the host computer 200. At this time, the scanning unit 110 projects optical pattern images to an object and captures the images, and transmits multiply captured images to the host computer 200 by control of the control unit 110. And, a plurality of scanning units 110 may be arranged around an object, however the present invention is not limited in this regard. For example, a single scanning unit 110 may be moved around an object or a single scanning unit 110 may be fixed while an object may be rotated at a predetermined angle.

[38] In the host computer 200, the input unit 210 is used in inputting information of an object for scanning, a scanning execution order and an image processing order, the output unit 220 is used in displaying a scanning result of an object, i.e. a three- dimensional image of the object, the database 230 is used in storing multiply captured image data 231 transmitted from the three-dimensional scanning apparatus 100, three- dimensional preprocessed image data 232 converted from the multiply captured image data 231 by STL (Standard Tesselation Language), three-dimensional image data 233 formed by converting the three-dimensional preprocessed image data 232 into a solid image of the object by merging and correction, and a three-dimensional image processing program 234 for three-dimensional scanning control and three-dimensional image processing, the communication interface 240 is used in exchanging data

between the host computer 200 and the three-dimensional scanning apparatus 100, and the central processing unit 260 is used in controlling the input unit 210, the output unit 220 and the database 230 connected to the BUS 250 based on the three-dimensional image processing program 234 stored in the database 230.

[39] Steps in which the above-mentioned three-dimensional scanning apparatus scans an object and transmits a captured image of the scanned object to the host computer are described below with reference to FIGs. 2 to 6.

[40] Referring to drawings, as shown in FIG. 4, the three-dimensional scanning apparatus 100 includes a plurality of scanning units 111 to 114 arranged in all directions around an object 300, for example a sphere, and each of the scanning units 111 to 114 has an optical projector (not shown) for projecting optical pattern images of a predetermined wavelength range to the object 300 according to a scanning manner, and a sensor camera (not shown) for capturing optical pattern images generated on surfaces of the object 300. Here, a plurality of the scanning units 111 to 114 each divides the object 300 into multiply areas (Ito IV) and scans the object 300 while moving vertically (SI lO).

[41] A plurality of the scanning units 111 to 114 project optical pattern images of a plurality of frames to the object 300, which are converted into horizontal or vertical shades at each of the multiply areas (Ito IV) and at each of vertical regional positions of the object 300, as shown in FIGs. 5 and 6, and capture a plurality of stripe images generated along the surfaces of the object 300. At this time, the control unit 120 of the three-dimensional scanning apparatus 100 determines whether scanning on the object 300 is completed (S 120).

[42] After scanning of the three-dimensional scanning apparatus 100 is completed, the sensor camera (not shown) of a plurality of the scanning unit 111 to 114 generates captured image data that the captured image for each of the areas (Ito IV) of the object 300 is converted into a binary digital signal by control of the control unit 120 (S 130), and transmits the generated captured image data to the host computer (S 140).

[43] In exemplary embodiments of the present invention, the digitalized captured image data for each of the areas (Ito IV) of the object 300 is automatically processed by a three-dimensional image processing program included in the host computer 200, and a three-dimensional image processing step is described below with reference to FIGs. 7 to 13.

[44] Referring to drawings, the three-dimensional image processing step includes a step

(S200) for converting the captured image data for each area of the object transmitted to the host computer into three-dimensional preprocessed image data, and a step (S300) for merging the three-dimensional preprocessed image data to form a single solid three-dimensional image data.

[45] The step S200 for converting the captured image data for each area of the object into three-dimensional preprocessed image data starts with a step (S210) for matching the image data for each area with a reference coordinate system to extract a three- dimensional coordinate value of the captured image for each area, and as shown in FIG. 8, and in the step S210, intersecting points 331 to 337 of the reference coordinate system 400 and the captured image of a predetermined area are extracted to calculate and extract a coordinate value defined as a three-dimensional vector value of (x.y,z).

[46] At this time, a reduced scale of the reference coordinate system 400 is matched with a setting space where the object is set in the three-dimensional scanning apparatus by a three-dimensional fixed grid system formed in the shape of a cubic. Specifically describing, the setting space is configured to a grid in a predetermined reduced scale, and thus the setting space is matched with the reference coordinate system 400. That is, setting the object in the setting space means setting the object in the reference coordinate system 400 matched structurally.

[47] Next, as shown in FIG. 9, the three-dimensional coordinate value of a predetermined area is converted into a point cloud 330, i.e. a set of points (S220), and as shown in FIG. 10, the point cloud 330 is converted into three-dimensional data (STL) of a mesh type having a plurality of polygons (S230). At this time, the polygon is a minimum unit expressing a solid shape.

[48] Subsequently, captured image data of other areas is sequentially converted into three-dimensional data, and as shown in FIG. 11, captured image data of other areas is matched with the reference coordinate system to extract a three-dimensional coordinate value of captured image for each area (S210), the three-dimensional coordinate value is converted into a point cloud 330, i.e. a set of points (S220), and the point cloud 330 is converted into three-dimensional data (STL) (S230).

[49] In exemplary embodiments of the present invention, the step S300 for merging three-dimensional data for each area of the object to convert into three-dimensional image data starts with a step (S310) for registering three-dimensional data for each area to arrange it on a coordinate system matched with the reference coordinate system. For example, as shown in FIG. 12, among three-dimensional data for each area of the object, three-dimensional data 330 and 340 of two areas are registered on a coordinate system. At this time, it is obvious that three-dimensional data of the other area is registered on a coordinate system, and thus is consistent in position.

[50] Next, the step S300 performs a step (S320) for merging the registered three- dimensional data for each area of the object to form a single three-dimensional image data. Here, as shown in FIG. 12, when three-dimensional data for each area is merged, an overlapping area (A) is generated between the three-dimensional data. At this time, in the overlapping area, an interface area is regenerated which has been removed and

of which an interface has been corrected in the merging step. That is, in the case that three-dimensional data configured in the shape of a plurality of polygons are overlapped in the overlapping area (A), an image is stained or deformed, and thus three-dimensional data is removed in the overlapping area (A), a three-dimensional data value adjoining the removed area is estimated and its interface is corrected, thereby regenerating an interface area.

[51] Subsequently, the step S300 performs a step (S330) for decimating the merged three-dimensional image data to quantify data. That is, in the step S330, a mesh number displayed in pixel is optimized while the original shape of the object configured in three-dimensional image is maintained. The optimization improves a three-dimensional image processing speed of the host computer.

[52] Next, the step S300 performs a step (S340) for hole filling the decimated three- dimensional image data to correct the overlapping area, an indented area and a blind spot between the three-dimensional data. At this time, the areas are corrected by decreasing or increasing the number of pixel or polygon under the estimation of an adjoining data value, as shown in FIG. 14, thus a solid image 300' of the object is completed, which is converted into a single three-dimensional image data.

[53] The above-mentioned three-dimensional image processing step is automatically performed by the host computer, such that the object is scanned, the scanned image data is processed and automatically converted into a solid image of the object with rapidity.

[54] As such, exemplary embodiments of the present invention have been described in detail with reference to the accompanying drawings. However, it should be understood that the detailed description and specific examples, while indicating preferred embodiments of the invention, are given by way of illustration only, since various changes and modifications within the spirit and scope of the invention will become apparent to those skilled in the art from this detailed description. Industrial Applicability

[55] As described above, the present invention automatically converts captured image for each area of an object transmitted from a three-dimensional scanning apparatus into a single three-dimensional image data by a host computer, thereby providing a user with improved working conditions and allowing the user to easily obtain three- dimensional image of the object.

[56] And, the present invention performs registering, merging, decimation and hole filling steps of three-dimensional preprocessed image data, thereby improving precision and perfection of three-dimensional object image.

[57]