Title of Invention: A CAMERA SYSTEM FOR FILMING GOLF

GAME AND THE METHOD FOR THE SAME

Technical Field

[0001] The present invention relates to field of golf game, particularly relates to a camera system and a method for filming a golf game.

Background Art

[0002] Videos taken during a golf game are good for entertainment as well as a tool to

improve golfer's skill. Generally, a video of a golf game is taken by another person (generally not the golfer to play the golf game) using a camera or a smart device with a built-in camera to trace the ball. Golf balls generally fly at a high speed, and it is difficult by human visual detection to trace a ball from a hit until the ball lands.

Therefore, it is desired to provide a camera which can automatically film golfer's swing and flying of a ball all the way until the ball lands. In existing technology, to determine the direction where the ball goes after hit, will be done manually. Firstly, user needs to turn the camera facing where he expects the ball to land after his hit and camera will memorize the direction. Afterward, camera will consequently turn to locate golfer. Once ball being hit and detected, the camera will turn from golfer to the recorded direction and film the video when ball lands.

Technical Problem

[0003] An aspect of the present invention is to provide a camera system for filming a golf game automatically. By utilizing imaging processing technique, camera system can capture the golfer and his/her related swing, detecting the ball and direction of hit including velocity of the ball, and film the golf ball landing without tracing the ball. Solution to Problem

Technical Solution

[0004] In training lesson of golfer, coach will always teach student to use two legs standing in parallel to the target line of hit. That is the stand of two legs always in parallel to the target line of hit. Camera system utilize this golf basic step to find the direction of the ball after hitting.

[0005] In one embodiment of the invention, the camera module comprises a single camera.

The single camera is a zoom camera and includes a zoom lens. The zoom levels of the camera are controlled according to the velocity of the ball.

[0006] In another embodiment of the invention, the camera module includes two cameras.

One is with prime lens and the other is with optical zoom lens. By placing the camera system in front of the golfer hitting location and with prime lens camera facing the ball on the ground, the algorithm can detect the ball through some image object recognition techniques. After scanned the location of ball, it will wait for golfer to walk in the ball near-by position so that the image of his/her legs can be identified accordingly and automatically. In the process, it measures the distances between ball and legs, the algorithm can estimate the angle that the camera must turn in order to parallel with the target line of hit. Once the ball is detected being hit by the prime lens, the camera with zoom lens will turn towards the target line of hit to film the ball landing scene until the ball stop rolling. Zoom lens will zoom with different levels according to the estimated velocity of the ball.

[0007] In another embodiment of the invention, only prime (fixed) lens without zoom lens are used. There can be an array of 3 lens which are in combination of prime lens and wide angle lenses with different focal lengths. To cover the range of golf ball flight, it can be divided into three 3 ranges. As the camera system can detect the velocity of ball being hit, hence the distance of the ball fly can be estimated. With the distance known, the camera system will activate the corresponding camera to capture the video accordingly.

Advantageous Effects of Invention

Advantageous Effects

[0008] The end result of the captured video is a complete record of a golfer's hit; from

swing to ball landing. It applies to different situation of golf during golf play; from Tee off with a driver, wood/iron hit on fairway, short game on chipping, bunker shot and putting respectively.

Brief Description of Drawings

Description of Drawings

[0009] Fig 1A shows the camera module with one camera;

[0010] Fig IB shows the camera module with two cameras;

[0011] Fig 1C shows the camera module with three cameras;

[0012] Fig 2 shows the internal structure of the camera system;

[0013] Fig 3 shows the DTL calculation from 90 degrees with the golfer;

[0014] Fig 4 shows the DTL calculation from 45 degrees with the golfer;

[0015] Fig 5 shows the ball detected through sensitivity zone;

[0016] Fig 6 shows the launch angle in relation to golfer and velocity of the ball captured.

[0017] Fig 7 shows the example of DTL in relation with the legs of golfer;

[0018] Fig 8 shows the DTL in relation to actual target towards the landing zone.

Best Mode for Carrying out the Invention

Best Mode

[0019] It will be appreciated that for simplicity and clarity of illustration, where appropriate, reference numerals have been repeated among the different figures to indicate corresponding or analogous elements. In addition, numerous specific details are set forth in order to provide a thorough understanding of the embodiments described herein.

However, it will be understood by those of ordinary skill in the art that the embodiments described herein can be practiced without these specific details. In other instances, methods, procedures, and components have not been described in detail so as not to obscure the related relevant feature being described. The drawings are not necessarily to scale and proportions of certain parts may be exaggerated to better illustrate details and features. The description is not to be considered as limiting the scope of the embodiments described herein.

[0020] Only one definition that applies throughout this disclosure will now be presented.

[0021] The term "comprising" means "including", but not necessarily limited to, it

specifically indicates open-ended inclusion or membership in a so-described combination, group, series, and the like.

[0022] Embodiments of the present disclosure will be described with reference to the accompanying the drawings.

[0023] The present invention provides a camera system for filming a golf game. As shown in Figures 1A-1C, the camera system includes a camera module 100, a pin 200 and a pole 300. The pin 200 and the pole 300 are employed for inserting the camera module 100 on the ground. The camera system further comprises a pan-tilt head 12, a circuit board 13, and a power module 14. The camera module 100 includes at least one camera 11. The pan-tilt head 12 is pivotally connected to the camera module 100, and can rotate the camera module 100 horizontally and vertically due to the actuation of motors 121 of the pan-tilt head 12. The motors 121 may be servo motors. The power module 14 provides power to the camera module 100. The power module 14 may be one or more batteries. The camera module 100 and the pan-tilt head 12 are electrically connected to the circuit board 13. The camera module 100 may further include a transparent cover 19 to cover and protect the camera module 100. The transparent cover 19 may be plastic, and may be circular or cylindrical.

[0024] Referring to FIG. 2, the camera system further includes a central processing unit 15, a sensor unit 16, a GPS module 17, and a wireless communication unit 18 arranged on the circuit board 13. The sensor unit 16 senses direction, horizontal angle, and vertical angle of the camera module 100. The sensor unit 16 may be a gyroscope. The sound senor unit 19 is for golfer voice recording and voice commands. The wireless communication unit 18 may communicate the camera module 100 with a portable device 400 of a golfer. The wireless communication unit 18 may be a WIFI module or/and a Bluetooth module. The portable device 400 may be a smart phone or a tablet PC. The central processing unit 15 controls the operation of the camera module 100, for example, controls the pan-tilt head 12 to pan the camera module 100 to a desired hitting direction so as to film the golf game with a preset proportion of zoom, and controls the communication between the camera module 100 and the portable device 400.

[0025] Concept of Direct Target Line (DTL) is illustrated in detail with Fig 7 and 8. During addressing, golfer stands in parallel to where he wants to hit the ball. By connecting two points of the golfer, i.e., two legs, two feet, two knees or other two positions of the golfer towards the target, the DTL is formed. Preferably, the two points are two legs of the golfer. Most balls will fly towards the DTL direction for skilled golfers. The camera module will film towards this direction after ball being hit, no matter the ball flies in line with DTL or not. Ball landing locations normally deviated from the DTL a bit, either left or right. But the zoom lens, with wide coverage of view, can film ball landing in a certain range derivate from DTL.

[0026] For skilled golfers, they may play the ball intentionally more towards right or left e.g. draw and slice the ball etc. The DTL is the final landing direction the golfer expects the ball to fly. It can be different from his final two-legs stand location. The camera system only scan his/her first legs location to determine the angle for camera to turn to. After recording the angle, golfer can stand differently from the DTL to perform draw and slice actions.

[0027] The present invention also provides a method for filming a golf game, in accordance with an exemplary embodiment of the camera system in the present invention.

[0028] In step 300, camera is firstly inserted on the ground facing the direction of golf ball in hitting area. In step 301, the ball is being scanned in the camera images and its location is determined. In step 302, golfer walks into the hitting area that image is being scanned and golfer body is determined. In Step 303, camera determines the legs of the golfer. According to the location of left and right legs as well as the ball, three vertical lines are drawn (see FIG.3 & 4). Dl is the distance between the right leg and ball as seen in camera view. Similarly, D2 is the distance between ball and the left leg. Based on ratio of Dl and D2, the DTL (Direct Target Line) angle as seen in bird view can be determined.

[0029] For example, in FIG.3, when the camera faces nearly perpendicular to the golfer, i.e. nearly perpendicular to the DTL, Dl is almost equal to D2 which means the target direction of ball flight is around 80 degrees from the current angle of the camera. The ratio for Dl and D2 will be positive and value is closing to 1. Similarly, in FIG.4, if the camera is placed with DTL angle 45 degree, the target direction of ball flight will be around 45 degrees from the current camera angle. The ratio of Dl and D2 will be in negative value indicating the DTL angle will be smaller than the Fig. 3 case.

[0030] Sheet 1 shows the relationship of DIAnd D2 and its ratio and DTL angle. Sheet 1

[Table 1]

[0032] In step 304, golfer hits the ball and camera images are being analyzed to determine if the ball has been successfully hit by the golfer. FIG.5 illustrates the detection of ball hit through a sensitivity zone, which is an image ROI (region of interest) where high framerate images are being monitored. The velocity and launch angle of the ball can be determined by comparing the sequence of ball images captured in the sensitivity zone (see FIG.6). With velocity measured, through Sheet 2, it can determine the duration of ball flight and distance of the ball travelled.

[0033] In step 305, the pan-tilt head 12 is controlled to pan the camera module 100 to the hitting direction parallel to the angle of the DTL calculated in step 303 so as to film the ball flying and landing, without actually tracing the ball.

[0034] Sheet 2

[] [Table 2]

[0035] In step 306, filming stops when a flight duration of the ball corresponding to the hitting distance elapses. In this embodiment, the camera module 100 stores relationships (Sheet 2) recording flight duration and velocity of the ball and hitting distances. Each range of hitting distance corresponds to one flight duration. For example, if the range of hitting distance is from 150 yards to 180 yards, the velocity is measured to be V6, the flight duration of the ball will be 6 seconds. The relationships are obtained according to the golfer's experience.

[0036] Embodiment 1

[0037] Referring to FIG.1A, the camera module 100 comprises a single camera 11. The single camera 11 is a zoom camera and includes a zoom lens 111. The zoom levels of the camera 11 are controlled according to the velocity of the ball after hitting as calculated from step 304. Sheet 3 shows the relationship of velocity to zoom level. E.g. when ball velocity V is calculated in the range of V2-V3, the zoom lens 111 will be zoomed at 1.5 X which is 1.5 times of the zoom levels.

[0038] Embodiment 2

[0039] Alternatively, in FIG IB, two cameras can be used, one is a prime lens or fish eye lens 211 while the other is a zoom lens 212. It can be configured to carry out the video task of golfer's swing and ball landing separately. The prime lens 211 is used to perform image processing for golfer image, ball velocity, and angle for the zoom lens 212 to turn, while the zoom lens 212 is used to take video of golfer's swing and ball landing.

[0040] The zoom levels of the zoom lens 212 are also controlled according to the velocity of the ball after hitting as calculated from step 304, which is similar to the zoom levels of the camera 11 in embodiment 1.

[0041] Embodiment 3

[0042] Only use prime (fixed) lens without zoom lens. There can be an array of 3 lens (A, B C) which are in combination of prime lens and wide angle lenses with different focal lengths. To cover the range of golf ball flight, it can be divided into three (3) ranges e.g. A for 1-100 yards; B for 100-200 yards; C for over 200 yards etc. As the camera system can detect the velocity of ball being hit, hence the distance of the ball fly can be estimated. With the distance known, the camera system will activate the corresponding camera to capture the video accordingly, e.g. if the ball detected flying distance is 160 yards, the camera B will be activated to capture the video.

[0043] Sheet 3 [Table 3]

[0044] The present invention also provides a method to determine the golf ball location, golfer body and golfer' s legs and generate a DTL. Given that the golfer is standing in front of the ball and with a typical preparation stand during addressing, it is known that the initial addressing (preparation stand) of golfer have a strong and known relationship to his/her intended target direction of ball traveling after hit. By capturing a sequence of images of this scene (at the preparation stage of the golfer hitting the ball), it can estimate the intended target direction of the ball, i.e. DTL (Direct Target Line), with respect to the camera view.

[0045] A method to determine the golf ball location, golfer body and golfer's legs and

generate a DTL comprises following steps:

[0046] determining the location of the ball in the scene by using multi-model image correlation;

[0047] determining the location of the golfer by using Background Modeling;

[0048] cleaning the model of golfer;

[0049] using the golfer model to calculate the distance between the legs of the golfers and to determine middle of two legs;

[0050] mapping the output of the relationship function to the DTL angle with a per- determined mapping function.

[0051] Therein,

[0052] 1. Ball localization using multi-model image correlation

[0053] building of the ball models: capturing image of balls at different scale and lighting conditions m(l,s), wherein 1 and s is lighting and scale parameter respectively;

[0054] using the correlation function coff(im, m(l,s)), coff is a template matching function, and search for the location with minimum coff with the use of all the ball models. This implies the location with the minimum values will be considered as the highest possible location of golf ball.

[0055] 2. Localization of the golfer using Background Modeling

[0056] After the location of the ball in the image is determined, based on this particular location in the image, the neighborhood area around the location of the ball will be selected, the size of the selected area will be subject to the size of the ball in the image. This area will be modeled based on the model pixel value of all location under stable condition.

[0057] First model is the one with background and without the golfer M _wg

[0058] Second model is the one with background and with the golfer M _g

[0059] Both models are built based on the temporal filtering, when the image is stable or without any change, the machine records the image model.

[0060] Finally, by subtracting the M _wg by M _g , then the result will be the model of golfer.

[0061] 3. Segmented region filtering

[0062] The result from the process of step 2 is the model of golfer, however it is required to eliminate some artifacts caused by noise and shadows. The small blob/regions are eliminated by morphological dilation operation.

[0063] Then edge detection is performed on the image, then the unwanted one is eliminated based on threshold of its orientation.

[0064] 4. Estimation of the position of golfer's legs

[0065] The result from the process of step 3 is a set of edges that reflected the vertical line of the golfer's legs. The process approximates the location of the pant by measuring the occurrence of the vertical line along the horizontal side of the camera view.

[0066] There should be three peaks if we plot the occurrence of edges against the x-direction of image space. Then using some histogram peak finding techniques to search for the local peaks. In our implementation, we are using hill-climbing techniques.

[0067] 5. Relationship function rel(p, b) between Golfer's stand position and ball

[0068] Relationship function is with the input, position of the legs, p, and position of the ball, b. The function itself is based on the ratio between the width of the legs and distance from the ball to the middle of the legs.

[0069] 6. Mapping function d(rel) maps the relationship rel(p, b) to turning angle DTL

[0070] In this process, the machine will map the rel value to the turning angle. The mapping function is determined by a combination of theoretical and empirical method.