FIELD

The invention relates to tracking an object.

BACKGROUND

Various exercises entail lifting a barbell carrying weight plates. These movements generally entail moving the barbell in a plane transverse to its length.

It can be useful to track the movement of the barbell during such exercises. Such analysis may reveal that an athlete produces poor vertical velocity, which may suggest that there is room to improve their technique. Similarly, tracking the horizontal movement of the barbell (in a direction transverse to its length) can also reveal details that may help the athlete improve.

"Velocity-based training" (VBT) is a term of art used loosely to encapsulate these various techniques, regardless of whether the end result returned to a user is an indication of velocity or, for example, a simple plot of the path that the barbell followed during a particular lift.

A range of technologies has previously been employed for VBT, including mechanical devices, such as cables that are pulled out from a floor-mounted reel, and optical techniques.

More recently, smart phones (such as the iPhone™) have been used. By way of example, the following video describes a My Lift app: "Tutorial & Validation: My Lift TRACKING JSS 2020", YouTube, uploaded by Carlos Balsalobre, 17 March 2020, <https://www.youtube.com/watch?v=WGU4VR8efzQ>. This app is configured to track the movement of a barbell based on video taken from the side of the athlete (i.e. looking at the barbell end-on). The user identifies the barbell by moving an onscreen circle into register with the weight plate and this calibration is then used to track the movement of the barbell. Traditional object-tracking techniques at the time of writing may blend a mixture of methods such as colour blob, edge extraction or movement detection (e.g. threshold difference from frame to frame). These methods may change dynamically, depending on the lighting conditions and the content of the footage. Many conventional techniques place a box about the tracked object as part of the calculation process (although the box might not be displayed to a user).

The present inventor has recognised that these apps are not as accurate as they might be, and that moving the on-screen circle to calibrate the app is an inconvenience for the user.

More specifically, the present inventor has recognised that inaccuracy can arise during the initial calibration procedure and also from the changing perspective as the weight plate (or set of weight plates) on the barbell is moved with respect to the camera. By way of example, a heavily loaded barbell is effectively loaded with a cylinder of weight plates at each end. As this barbell is lifted above the camera, the lower portion of the rearmost weight plate may come into view whereby conventional object tracking techniques may underestimate the extent to which the barbell is lifted.

With the foregoing in mind, the present invention aims to provide improvements in and for tracking a circular weight plate, or at least to provide an alternative for those concerned with such tracking.

It is not admitted that any of the information in this patent specification is common general knowledge, or that the person skilled in the art could be reasonably expected to ascertain or understand it, regard it as relevant or combine it in any way before the priority date.

SUMMARY

One aspect of the invention provides a method, of tracking a circular weight plate, comprising a computer searching each frame of a plurality of frames for one or more ellipses. Preferably, the method comprises finding, amongst the plurality of frames, multiellipse frames, in which case the method may comprise mutually comparing ellipses within each respective frame of the multi-ellipse frames.

As the wording is used herein, a multi-ellipse frame is a frame comprising three or more ellipses.

Preferably, the method comprises dividing ellipses within each respective frame of the multi-ellipse frames into a respective plurality of groups, by mutually comparing, the ellipses within each respective frame of the multi-ellipse frames. Each group may substantially consist of ellipses in which each parameter of a set of parameters varies within defined limits, the set of parameters comprising two or more of horizontal position, vertical position, major axis, minor axis and angular orientation. Preferably, the set of parameters comprises two or more of horizontal position, vertical position, major axis and minor axis. The method may comprise selecting candidate groups. Optionally, each candidate group is a respective largest group of a respective plurality of groups and/or comprises one or more ellipses from a respective frame.

The method may comprise smoothing over time at least one parameter of ellipses, e.g. of ellipses of the candidate groups. The at least one parameter may be vertical position. Preferably the smoothing over time comprises fitting a cubic spline.

The searching each frame of a plurality of frames for one or more ellipses may be searching each frame of a plurality of frames for one or more large ellipses. The searching each frame of a plurality of frames for one or more ellipses may be searching each frame of a plurality of frames for one or more complete ellipses.

The method may comprise scaling based on a major axis of an ellipse, preferably based on major axes of two or more ellipses.

The method optionally comprises producing the frames. The producing may comprise frame differencing. Preferably, the producing the frames comprises edge extraction. Optionally, it comprises image sharpening. The producing may comprise camera recording. Alternatively, a set of pre-recorded frames may be supplied as an input to the method.

The computer is preferably a smart phone.

Another aspect of the invention provides a computer configured to track a circular weight plate.

Another aspect of the invention provides a computer-readable medium carrying instructions computer-executable to cause a computer to track a circular weight plate.

BRIEF DESCRIPTION OF DRAWINGS

Figure 1 is a flowchart charting a method of tracking;

Figure 2 is a video frame of a bench press exercise;

Figure 3 shows the video frame after pre-processing;

Figure 4 is a scatter plot of vertical position with respect to time; and

Figure 5 illustrates a candidate group of two ellipses.

DESCRIPTION OF EMBODIMENTS

Figure 2 shows a frame from the first step of the method set out in Figure 1 . Within the frame is an operator performing a bench press. This exercise entails a barbell 1 carrying a near weight plate 3a and a far weight plate 3b. Also within the frame are numerous other ellipses such as the ellipses up and down the rack 5 and other weight plates 7a, 7b, 7c and the roman rings 9.

The video feed is preferably obtained utilising a camera holder such as a tripod. Most preferably the camera is oriented to approximately centre in each frame a centre of a range of motion of the weight plate of interest. The raw video feed is pre-processed to produce the frame illustrated in Figure 3. The pre-processing preferably comprises image sharpening, frame differencing and edge & contour detection. Edges which form contiguous contours are highlighted. Any convenient contour detection algorithm may be used, though the Suzuki-Abe method is preferred (Suzuki, Satoshi and Abe, Keiichi, “Topological structural analysis of digitized binary images by border following”, in Science Direct on Computer Vision, Graphics, and Image Processing, vol 30, issue 1 , pp. 32-46, Apr. 1985). Sharpening helps overcome motion blur when the weight is lifted fast. Comparing frames for difference means subsequent ellipse detection ignores static objects in the frame. The frame differencing may entail subtracting from one frame the pixels from a previous frame. Any convenient edge extraction algorithm may be used. Methods based on the Deriche method are preferred. Other options include the Canny method, or newer deep learning methods.

The preferred pre-processing reduces the frame to a set of contours corresponding to the features that have moved since a previous frame. The barbell 1 and weight plates 3a, 3b are evident as are portions of the athlete’s hands and arms. For the sake of clarity, Figure 3 herein is shown as black on white whereas more usually the output from frame differencing is displayed as white on black.

The video recording and pre-processing steps may form part of the weight platetracking method. Alternatively, these steps might be performed separately and supplied as an input to a weight plate-tracking method.

According to the preferred approach, a series of frames akin to the frame of Figure 3 are each searched for ellipses. Any convenient ellipse-detecting algorithm may be used, although the Jia method is preferred (Q. Jia, X. Fan, Z. Luo, L. Song and T. Qiu, 'A Fast Ellipse Detector Using Projective Invariant Pruning," in IEEE Transactions on Image Processing, vol. 26, no. 8, pp. 3665-3679, Aug. 2017).

The search for ellipses is preferably biased towards locating large ellipses. This may be achieved by selecting a certain proportion, e.g. the largest 25%, of the ellipses detected. It could also entail obtaining only ellipses of at least a minimum size (pixel count), e.g. minimum major axis. In preferred implementation the search discards ellipses having major axes of less than 5% of the frame, so for 1080px video width, ellipses smaller than ~50px are rejected. Minimum relative size (e.g. at least 90% as large as the largest ellipse found in the frame) is another option. Other criterion may also be applied as part of an effort to locate ellipses corresponding to the near weight plate 3a. By way of example, eccentricity might be taken into account. Arc-fitting ellipse detection methods, such as the Jia method, can be implemented with defined minimum arc radii whereby smaller and/or eccentric ellipses are filtered out in this way.

By focusing on large ellipses, smaller ellipses such as the end 1a of the barbell and inboard features of the weight plate can be ignored and larger ellipses approximately corresponding to the outer periphery of the near weight plate 3a can be located. It is also desirable to focus on complete ellipses (i.e. focus on ellipses that do not extend partly out of the frame).

For some implementations of the method, the ellipse-searching process might be tuned to seek out only one ellipse which should correspond to the true position of the near weight plate 3a. In practice, tuning the process to return a single accurate ellipse is problematic due to errors associated with image blur and lighting changes, etc. Accordingly, it is preferred to locate multiple potential ellipses in each frame and then apply subsequent processing steps to find the true position of the weight plate based on this plurality of results.

A preferred approach entails grouping the located ellipses on a frame-by-frame basis. The grouping may be based on any convenient means. By way of example, the ellipses of one frame might be divided into a predetermined set of size categories for each frame. Preferably, the grouping entails mutually comparing ellipses, e.g. the assessment of similarity may be based on at least one, or more preferably at least 4, e.g. all five, of horizontal position, vertical position, major axis, minor axis and angular orientation. For the avoidance of doubt, grouping ellipses based on a trigonometric function of their angular orientation is substantially equivalent to grouping based on angular orientation per se, as the wording is used herein. Most preferably, the assessment of similarity is in accordance with one or more of the methods described by Prasad and Leung (D. K. Prasad and M. K. H. Leung, "Clustering of ellipses based on their distinctiveness: An aid to ellipse detection algorithms", 2010 3 ^rd International Conference on Computer Science and Information Technology, 2010, pp. 292-297).

Following the preferred approach, the largest group (that is, the group having the most ellipses) of each frame is selected, although other selection criteria are possible. The inventor has recognised that typical edge extraction techniques usually reveal more ellipses for large features than for smaller ones whereby the largest group of similar ellipses reliably corresponds to the largest circular object moving in the frame. A parameter such as vertical height of these ellipses can then be smoothed over time. Figure 4 illustrates a preferred approach. The vertical positions of the selected ellipses are plotted over time as a scatter plot. The data can then be smoothed using a smoothing spline or other convenient technique. Optionally, data from any one-ellipse or two-ellipse frames may be included in the scatter plot and smoothing operation.

Figure 4 is a plot of vertical position over time. A similar smoothing process might be applied to data showing horizontal position (i.e. x-axis) over time. In yet another variant, instead of separately smoothing horizontal and vertical position data over time, a single three-dimensional scatter plot of the two positions over time might be smoothed. Tracking the z-axis may have utility in a variant of the concept. The x-axis is a horizontal axis generally perpendicular to an axis of the barbell. The z-axis is a horizontal axis generally parallel to an axis of the barbell.

The present inventor has recognised that the major axis of the ellipse corresponding to the weight plate 3a corresponds to the diameter of the weight plate 3a in reality, and that a projection of a flat disk on a camera image as an ellipse has unique parameters which can be used to establish the disk's real world distance and position in space relative to the lens of the camera. These relationships can be used to scale pixel count data to real world measurements of distance, e.g. a measurement of the distance that the bar is lifted from an initial starting position. Olympic-sized weight plates have a diameter of 450 mm. This figure may be stored as a default weight plate size. Optionally, the user may be given an opportunity to enter a weight plate diameter. The inventor has also recognised that the relationship between pixel count and actual size changes as an object is moved through the frame as a result of changing perspective, etc. Accordingly, the scaling preferably takes account of the major axis of the relevant ellipse in at least two frames, or more preferably in a number of frames including one frame at about a bottom of a lift and one frame at about a top of a lift.

Preferably, the data is scaled prior to the smoothing step although potentially these two steps could be performed in the reverse order.

The scaling may entail scaling multiple ellipses in each individual frame of a plurality of frames. According to the preferred approach, on a frame by frame basis, a position for each ellipse of the selected group (in this case largest group) of ellipses is scaled to an estimated actual position. The scaling is preferably with respect of a centre of the frame. Figure 5 illustrates a selected group of two ellipses, ellipses A and B, in a frame. The figure also illustrates pixel counts the major axes and vertical positions with respect to central point y=0. For the sake of clarity, the differences between ellipses A and B have been exaggerated. In practice, ellipses within a group are often within a few pixels of each other. The actual vertical positions of ellipses A and B may be estimated as follows:

For each ellipse:

- The 0450mm diameter is divided by the major axis pixel count to determine a conversion factor; and

The vertical position pixel count is multiplied by the conversion factor to estimate the actual vertical position. In theory, there might be room for improvement by integrating rather than multiplying. In practice when using the camera of a modern smart phone this simple scaling operation followed by a smoothing operation has been found to be noise tolerant and to produce accurate results for both vertical position and horizontal position. By way of example, as a test, the inventor arranged two smart phones at different orientations to track the same weight plate through an exercise without appreciable differences between the two sets of results.

Many variants of the concept are possible. Whilst preferably each and every frame of a video feed is analysed, in principle, satisfactory results might be obtained by analysing only every second or third frame, for example. In a preferred implementation, the computer is a smartphone, although other forms of computer such as a distributed computer, are possible. Most preferably, the same smart phone video-records the original video, pre-processes that video feed and searches for ellipses amongst the pre-processed image. These steps might occur sequentially but may also occur on the fly in parallel to each other and essentially in real time, e.g. the smoothing step might commence as soon as the characterising ellipse parameters (e.g. major axes and pixel counts for vertical position) are determined. Indeed, the method might be implemented without retaining a video feed as such; rather, frames from the smart phone’s camera might be analysed and only the resultant smoothing spline recorded. Tracking the z-axis may have utility in a variant of the concept.

The smoothing splines might be displayed to a user "as-is" to show displacement of the weight plate in either axis. Alternatively, the data from these two axes plot the path of the barbell against x/y coordinates. An animation might show the barbell's movement along the path, e.g. in slow motion. Logic might be applied to the data to highlight key features to a user, such as velocity maxima and/or or minima.

Alternatively, the displacement measurement can be interpreted for end users as measurements, such as range of motion, velocity (mean, peak, accelerative, etc), time under tension, movement tempo, barbell path and so on, or combined with the amount of weight being lifted to calculate measurements of force like peak power. These derivative measurements can be calculated after the video is analysed, or while the analysis is in progress. This can inform coaching choices during an exercise (e.g. stop the exercise at a predetermined threshold of velocity), or between sets of the exercise (e.g. decide to increase or decrease load), and across multiple training sessions (e.g. planning long term load or velocity targets).

The methods described herein may be employed to track other circular objects. Indeed, yet other variants may be employed to track objects of a different shape. An ellipse is an example of a corresponding shape that appears in a video feed and corresponds to a circle in reality. Other corresponding shapes might be the target for tracking other shapes.

The invention is not limited to the examples discussed herein. Rather, the invention is defined by the claims.

The term "comprises" and its grammatical variants has a meaning that is determined by the context in which it appears. Accordingly, the term should not be interpreted exhaustively unless the context dictates so.