CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] The present application claims priority to U.S. App. Ser. No. 63/039,911 , filed on Jun. 16, 2020 and entitled “A RING DEVICE HAVING AN ANTENNA, A TOUCH PAD, AND/OR A CHARGING PAD TO CONTROL A COMPUTING DEVICE BASED ON USER MOTIONS,” and U.S. Pat. App. Ser. No. 17/347,557, filed on Jun. 14, 2021 and entitled “A DEVICE HAVING AN ANTENNA, A TOUCH PAD, AND/OR A CHARGING PAD TO CONTROL A COMPUTING DEVICE BASED ON USER MOTIONS,” the entire disclosures of which applications are all hereby incorporated herein by reference.

[0002] The present application relates to U.S. Pat. App. Ser. No. 16/576,661 , filed Sep. 19, 2019 and entitled “Calibration of Inertial Measurement Units in Alignment with a Skeleton Model to Control a Computer System based on Determination of Orientation of an Inertial Measurement Unit from an Image of a Portion of a User,” U.S. Pat. App. Ser. No. 16/044,984, filed Jul. 25, 2018 and entitled “Calibration of Measurement Units in Alignment with a Skeleton Model to Control a Computer System,” U.S. Pat. App. Ser. No. 15/973,137, filed May 7, 2018 and entitled “Tracking User Movements to Control a Skeleton Model in a Computer System,” U.S. Pat. App. Ser. No. 15/868,745, filed Jan. 11 , 2018 and entitled “Correction of Accumulated Errors in Inertial Measurement Units Attached to a User,” U.S. Pat. App. Ser. No. 15/864,860, filed Jan. 8, 2018 and entitled “Tracking Torso Leaning to Generate Inputs for Computer Systems,” U.S. Pat. App. Ser. No. 15/847,669, filed Dec. 19, 2017 and entitled “Calibration of Inertial Measurement Units Attached to Arms of a User and to a Head Mounted Device,” U.S. Pat. App. Ser. No. 15/817,646, filed Nov. 20, 2017 and entitled “Calibration of Inertial Measurement Units Attached to Arms of a User to Generate Inputs for Computer Systems,” U.S. Pat. App. Ser. No. 15/813,813, filed Nov. 15, 2017 and entitled “Tracking Torso Orientation to Generate Inputs for Computer Systems,” U.S. Pat. App. Ser. No. 15/792,255, filed Oct. 24, 2017 and entitled “Tracking Finger Movements to Generate Inputs for Computer Systems,” U.S. Pat. App. Ser. No. 15/787,555, filed Oct. 18, 2017 and entitled “Tracking Arm Movements to Generate Inputs for Computer Systems,” and U.S. Pat. No. 15/492,915, filed Apr. 20, 2017 and entitled “Devices for Controlling Computers based on Motions and Positions of Hands,” the entire disclosures of which applications are hereby incorporated herein by reference.

TECHNICAL FIELD

[0003] At least a portion of the present disclosure relates to computer input devices in general and more particularly but not limited to input devices for virtual reality and/or augmented/mixed reality applications implemented using computing devices, such as mobile phones, smart watches, similar mobile devices, and/or other devices. The described input devices may be used with other types of devices such as Internet-of- Things (loT) devices including smart home appliances, drone devices, autonomous and non-autonomous vehicles, and similar types of devices requiring user input.

BACKGROUND

[0004] U.S. Pat. App. Pub. No. 2014/0028547 discloses a user control device having a combined inertial sensor to detect the movements of the device for pointing and selecting within a real or virtual three-dimensional space.

[0005] U.S. Pat. App. Pub. No. 2015/0277559 discloses a finger-ring-mounted touchscreen having a wireless transceiver that wirelessly transmits commands generated from events on the touchscreen.

[0006] U.S. Pat. App. Pub. No. 2015/0358543 discloses a motion capture device that has a plurality of inertial measurement units to measure the motion parameters of fingers and a palm of a user.

[0007] U.S. Pat. App. Pub. No. 2007/0050597 discloses a game controller having an acceleration sensor and a gyro sensor. U.S. Pat. No. D772,986 discloses the ornamental design for a wireless game controller.

[0008] Chinese Pat. App. Pub. No. 103226398 discloses data gloves that use micro-inertial sensor network technologies, where each micro-inertial sensor is an attitude and heading reference system, having a tri-axial micro-electromechanical system (MEMS) micro-gyroscope, a tri-axial micro-acceleration sensor and a tri-axial geomagnetic sensor which are packaged in a circuit board. U.S. Pat. App. Pub. No. 2014/0313022 and U.S. Pat. App. Pub. No. 2012/0025945 disclose other data gloves. [0009] U.S. Pat. App. Pub. No. 2016/0085310 discloses techniques to track hand or body pose from image data in which a best candidate pose from a pool of candidate poses is selected as the current tracked pose.

[0010] U.S. Pat. App. Pub. No. 2017/0344829 discloses an action detection scheme using a recurrent neural network (RNN) where joint locations are applied to the recurrent neural network (RNN) to determine an action label representing the action of an entity depicted in a frame of a video.

[0011] U.S. Pat. App. Pub. No. 2017/0186226 discloses a calibration engine that uses a machine learning system to extracts a region of interest to compute values of shape parameters of a 3D mesh model.

[0012] U.S. Pat. App. Pub. No. 2017/0186226 discloses a system where an observed position is determined from an image and a predicted position is determined using an inertial measurement unit. The predicted position is adjusted by an offset until a difference between the observed position and the predicted position is less than a threshold value.

[0013] The disclosures of the above discussed patent documents are hereby incorporated herein by reference.

BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The embodiments are illustrated by way of example and not limitation in the figures of the accompanying drawings in which like references indicate similar elements.

[0015] FIG. 1 illustrates a system to track user movements according to one embodiment.

[0016] FIG. 2 illustrates a system to control computer operations according to one embodiment.

[0017] FIG. 3 illustrates a skeleton model that can be controlled by tracking user movements according to one embodiment.

[0018] FIG. 4 illustrates a technique to determine an orientation of a ring device for tracking user movements using an image captured by a camera on a head mounted display according to some embodiments.

[0019] FIG. 5 shows a ring device having components configured at locations according to some embodiments.

[0020] FIG. 6 shows another ring device according to some embodiments.

[0021] FIG. 7 and FIG. 8 illustrate an alternative ring device according to some embodiments of the disclosure.

[0022] FIG. 9 illustrates an alternative ring device in an opened and closed state according to some embodiments of the disclosure.

[0023] FIG. 10 is a photograph illustrating a prototype of a ring device according to some embodiments of the disclosure.

[0024] FIGS. 11-14 illustrate an alternative ring device employing a replicable “snail” connector mechanism according to some embodiments of the disclosure.

[0025] FIG. 15 is a diagram of an alternative ring design according to some embodiments of the disclosure.

[0026] FIGS. 16-23 are diagrams of an armband device and mount according to some embodiments of the disclosure.

[0027] FIG. 24 is a perspective view of a charging case for an armband module and/or ring device according to some embodiments of the disclosure.

[0028] FIG. 25 is a perspective view of an alternative charging case for an armband module and/or ring device according to some embodiments of the disclosure.

[0029] FIG. 26 is the top perspective view of the alternative charging case according to some embodiments of the disclosure.

[0030] FIG. 27 is a side view of the alternative charging case according to some embodiments of the disclosure.

[0031] FIG. 28 is a cutaway view of the alternative charging case illustrating the internal circuitry of the case according to some embodiments of the disclosure.

[0032] FIG. 29 is a front-bottom perspective view of the alternative charging case according to some embodiments of the disclosure.

[0033] FIG. 30 is a bottom view of the alternative charging case according to some embodiments of the disclosure.

[0034] FIG. 31 is a cutaway view of the alternative case according to some embodiments of the disclosure.

[0035] FIG. 32 is a perspective view of a lid portion of the alternative case according to some embodiments of the disclosure.

DETAILED DESCRIPTION

[0036] The following description and drawings are illustrative and are not to be construed as limiting. Numerous specific details are described to provide a thorough understanding. However, in certain instances, well-known or conventional details are not described to avoid obscuring the description. References to one or an embodiment in the present disclosure are not necessarily references to the same embodiment; and, such references mean at least one.

[0037] U.S. Pat. App. Ser. No. 16/044,984, filed Jul. 25, 2018, and entitled “Calibration of Measurement Units in Alignment with a Skeleton Model to Control a Computer System,” the entire disclosure of which is hereby incorporated herein by reference, discloses sensor modules having LED lights that can be used to provide optical indicators in the determination of the orientations of the sensor modules. A camera (e.g., in the head-mounted display) can be used to capture images of the optical indicators to determine the orientations of the sensor modules. After identifying the locations of LED lights of the sensor module in an image, the locations of the LED lights can be processed via an artificial neural network (ANN) to provide an orientation measurement for the sensor module. The orientation measurement for the sensor module, determined based on the optical indicators, can be used to calibrate orientation measurements generated by an inertial measurement unit in the sensor module. [0038] In some instances, the LED lights of the sensor module may not be in a position visible to the camera and thus cannot be captured as optical indicators in the images generated by the camera. In other instances, the sensor module may not have LED lights configured on the sensor modules. The present application discloses techniques that can be used to determine the orientation of the sensor module based on images captured in the camera without relying upon LED optional indicators. For example, when the sensor module is being held or worn on a portion of the user in a predetermined manner, an image of the potion of the user can be used in a first ANN to determine the orientation of predefined features of the user and then used in a second ANN to predict the orientation of the sensor module based on the orientations of the predefined features of the user. For example, the sensor module can be in the form of a ring worn on a predetermined finger of a hand of the user; and the first ANN can be used to determine the orientations of features of the user, such as the orientations of the wrist, palm, forearm, and/or distal, middle and proximal phalanges of thumb and/or index finger of the user.

[0039] For example, a sensor device can be configured as a ring attached to the middle phalange of the index finger; and the sensor device has a touch pad. When the thumb of the user is placed on the touch pad of the sensor device, the orientation of the sensor device can be predicted based on the orientations of the bones of the thumb and/or the index finger. Thus, in response to the configuration of the thumb being on the touching pad of the sensor device worn on the middle phalange of the index finger, an image of the hand can be provided as an input to an ANN to determine the orientations of certain features on the hand of the user, which orientations can be used in a further ANN to determine the orientation of the ring/sensor device. For example, the features identified/used for the determination of the orientation of the ring/sensor device can include bones and/or joints, such as wrist, palm, phalanges of thumb and index finger.

[0040] Once the orientation of the sensor device is determined, calibration can be performed in a way similar to those disclosed in U.S. Pat. App. Ser. No. 16/044,984, filed Jul. 25, 2018 and entitled “Calibration of Measurement Units in Alignment with a Skeleton Model to Control a Computer System,” the entire disclosure of which is hereby incorporated herein by reference. [0041] In general, uncalibrated measurements of an inertial measurement unit (IMU) can be considered as orientations of the inertial sensor measured relative to an unknown reference coordinate system. A calibration process identifies the unknown reference coordinate system and its relationship with respect to a known coordinate system. After the calibration, the measurements of the IMU are relative to the known coordinate system. For example, the calibrated measurements can be an orientation relative to a predetermined orientation in the space, relative to a particular orientation of the sensor device at a specific time instance, relative to the orientation of the arm or hand of a user at a time instance, or relative to a reference orientation/pose of a skeleton model of the user.

[0042] In some embodiments, the determination of calibration parameters of the measurements of the inertial measurement unit such that the calibrated measurements of the inertial measurement unit are relative to a known orientation, such as the orientation of the sensor device in which the inertial measurement unit is installed, the orientation of the arm or hand of a user to which the sensor device is attached, or the orientation of a skeleton model of the user in a reference pose. For example, a stereo camera integrated into a head mount display (FIMD) can be used to capture images of sensor modules on the user. In some embodiments, Computer vision techniques and/or ANN techniques can process the captured images identify one or more orientations that can be used to calibrate the measurements of the inertial measurement units in the sensor modules.

[0043] In general, the kinematics of a user can be modeled using a skeleton model having a set of rigid parts/portions connected by joints. For example, the head, the torso, the left and right upper arms, the left and right forearms, the palms, phalange bones of fingers, metacarpal bones of thumbs, upper legs, lower legs, and feet can be considered as rigid parts that are connected via various joints, such as the neck, shoulders, elbows, wrist, and finger joints.

[0044] The movements of the parts in the skeleton model of a user can be controlled by the movements of the corresponding portions of the user tracked using sensor modules. The sensor modules can determine the orientations of the portions of the user, such as the hands, arms, and head of the user. The measured orientations of the corresponding parts of the user determine the orientations of the parts of the skeleton model, such as hands and arms. The relative positions and/or orientations of the rigid parts collectively represent the pose of the user and/or the skeleton model. The skeleton model of the user can be used to control the presentation of an avatar of the user, to identify the gesture inputs of the user, and/or to make a virtual realty or augmented reality presentation of the user.

[0045] FIG. 1 illustrates a system to track user movements according to one embodiment.

[0046] FIG. 1 illustrates various parts of a user, such as a torso (101 ) of the user, the head (107) of the user, the upper arms (103, 105) of the user, the forearms (112, 114) of the user, and the hands (106, 108) of the user.

[0047] In FIG. 1, the hands (106, 108) of the user are considered rigid parts, movable around the wrists of the user. In other embodiments, the palms and finger bones of the user can be further tracked for their movements relative to finger joints (e.g., to determine the hand gestures of the user made using relative positions among fingers of a hand and the palm of the hand).

[0048] In FIG. 1 , the user wears several sensor modules/devices (111 , 113, 115, 117, 119) that track the orientations of parts of the user that are considered, or recognized as, rigid in an application.

[0049] As illustrated, rigid parts of the user are movable relative to the torso (101 ) of the user, and relative to each other. Examples of the rigid parts include the head (107), the upper arms (103, 105), the forearms (112, 114), and the hands (106, 108). The joints, such as neck, shoulder, elbow, and/or wrist, connect the rigid parts of the user to form one or more kinematic chains. The kinematic chains can be modeled in a computing device (141 ) to control the application.

[0050] To track the relative positions/orientations of rigid parts in a kinematic chain that connects the rigid parts via one or more joints, a tracking device can be attached to each individual rigid part in the kinematic chain to measure its orientation.

[0051] In general, the position or orientation of a rigid part in a reference system (100) can be tracked using one of many systems known in the field. Some of the systems may use one or more cameras to take images of a rigid part marked using optical markers and analyze the images to compute the position and/or orientation of the part. Some of the systems may track the rigid part based on signals transmitted from, or received at, a tracking device attached to the rigid part, such as radio frequency signals, infrared signals, ultrasound signals. The signals may correspond to signals received in the tracking device, and/or signals emitted from the tracking device. Some of the systems may use inertial measurement units (IMUs) to track the position and/or orientation of the tracking device.

[0052] In FIG. 1, the sensor devices (111 , 113, 115, 117, 119) are used to track some of the rigid parts (e.g., 107, 103, 105, 106, 108) in the one or more kinematic chains, but sensor devices are omitted from other rigid parts (101 , 112, 114) in the one or more kinematic chains to reduce the number of sensor devices used and/or to improve user experience for wearing the reduced number of sensor devices.

[0053] The computing device (141 ) can have a prediction model (141 ) trained to generate predicted measurements of parts (101 , 112, 114, 107, 103, 105, 106, or 108) of the user based on the measurements of the sensor devices (111 , 113, 115, 117, 119).

[0054] For example, the prediction model (141 ) can be implemented using an ANN in the computing device (141 ) to predict the measurements of the orientations of the rigid parts (101 , 112, 114) that have omitted sensor devices, based on the measurements of the orientations rigid parts (107, 103, 105, 106, 108) that have the attached sensor devices (111 , 113, 115, 117, 119).

[0055] Further, the ANN can be trained to predict the measurements of the orientations of the rigid parts (107, 103, 105, 106, 108) that would be measured by another system (e.g., an optical tracking system), based on the measurement of the attached sensor devices (111 , 113, 115, 117, 119) that measure orientations using a different technique (e.g., IMUs).

[0056] The sensor devices (111 , 113, 115, 117, 119) communicate their movement measurements to the computing device (141 ), which computes or predicts the orientation of the rigid parts (107, 103, 105, 106, 108, 101 , 112, 114) by applying the measurements obtained from the attached sensor devices (111 , 113, 115, 117, 119) as inputs to an ANN trained in a way as further discussed below.

[0057] In some implementations, each of the sensor devices (111 , 113, 115, 117, 119) communicates its measurements directly to the computing device (141 ) in a way independent from the operations of other sensor devices.

[0058] Alternative, one of the sensor devices (111 , 113, 115, 117, 119) may function as a base unit that receives measurements from one or more other sensor devices and transmit the bundled and/or combined measurements to the computing device (141 ). In some instances, the ANN is implemented in the base unit and used to generate the predicted measurements that are communicated to the computing device (141 ).

[0059] Preferably, wireless connections made via a personal area wireless network (e.g., Bluetooth connections), or a local area wireless network (e.g., Wireless-Fidelity, “Wi-Fi,” connections) are used to facilitate the communication from the sensor devices (111 , 113, 115, 117, 119) to the computing device (141 ).

[0060] Alternatively, wired connections can be used to facilitate communication among some of the sensor devices (111 , 113, 115, 117, 119) and/or with the computing device (141).

[0061] For example, a hand module (117, 119) attached to or held in a corresponding hand (106, 108) of the user may receive the motion measurements of a corresponding arm module (113, 115) and transmit the motion measurements of the corresponding hand (106, 108) and the corresponding upper arm (105, 103) to the computing device (141 ).

[0062] The hand (106), the forearm (114), and the upper arm (105) can be considered a kinematic chain, for which an ANN can be trained to predict the orientation measurements generated by an optical track system, based on the sensor inputs from the sensor devices (117 and 115) that are attached to the hand (106) and the upper arm (105), without a corresponding device on the forearm (114).

[0063] Alternatively, or in conjunction with the foregoing, the hand module (e.g., 117) may combine its measurements with the measurements of the corresponding arm module (115) to compute the orientation of the forearm connected between the hand (106) and the upper arm (105), in a way as disclosed in U.S. Pat. No. 10,379,613, entitled “Tracking Arm Movements to Generate Inputs for Computer Systems,” the entire disclosure of which is hereby incorporated herein by reference.

[0064] For example, the hand modules (117, 119) and the arm modules (113, 115) can be each respectively implemented via a base unit (or a game controller) and an arm/shoulder module discussed in U.S. Pat. No. 10,509,469, entitled “Devices for Controlling Computers based on Motions and Positions of Hands,” the entire disclosure of which application is hereby incorporated herein by reference.

[0065] In some implementations, the head module (111 ) is configured as a base unit that receives the motion measurements from the hand modules (117, 119) and the arm modules (113, 115) and bundles the measurement data for transmission to the computing device (141 ). In some instances, the computing device (141 ) is implemented as part of the head module (111 ). The head module (111 ) may further determine the orientation of the torso (101 ) from the orientation of the arm modules (113, 115) and/or the orientation of the head module (111 ), using an ANN trained for a corresponding kinematic chain, which includes the upper arms (103, 105), the torso (101 ), and/or the head (107).

[0066] For the determination of the orientation of the torso (101 ), the hand modules (117, 119) are optional in the system illustrated in FIG. 1. Further, in some instances, the head module (111 ) is not used in the tracking of the orientation of the torso (101 ) of the user.

[0067] Typically, the measurements of the sensor devices (111 , 113, 115, 117, 119) are calibrated for alignment with a common reference system, such as the coordinate system (100). For example, the coordinate system (100) can correspond to the orientation of the arms and body of the user in a standardized pose illustrated in FIG. 1. When in the pose of FIG. 1, the arms of the user point in the directions that are parallel to the Y axis; the front-facing direction of the user is parallel to the X axis; and the legs, the torso (101 ) to the head (107) are in the direction that is parallel to the Z axis.

[0068] After the calibration, the hands, arms (105, 103), the head (107), and the torso (101 ) of the user may move relative to each other and relative to the coordinate system (100). The measurements of the sensor devices (111 , 113, 115, 117, 119) provide orientations of the hands (106, 108), the upper arms (105, 103), and the head (107) of the user relative to the coordinate system (100). The computing device (141 ) computes, estimates, or predicts the current orientation of the torso (101 ) and/or the forearms (112, 114) from the current orientations of the upper arms (105, 103), the current orientation the head (107) of the user, and/or the current orientation of the hands (106, 108) of the user and their orientation history using the prediction model (116).

[0069] Some techniques of using an ANN to predict the movements of certain parts in a skeleton model that are not separately tracked using dedicated sensor devices can be found in U.S. Pat. App. Ser. No. 15/996,389, filed Jun. 1 , 2018 and entitled “Motion Predictions of Overlapping Kinematic Chains of a Skeleton Model used to Control a Computer System,” and U.S. Pat. App. Ser. No. 15/973,137, filedMay 7, 2018 and entitled “tracking User Movements to Control a Skeleton Model in a Computer System,” the entire disclosures of which applications are hereby incorporated herein by reference.

[0070] Optionally or in combination, the computing device (141 ) may further compute the orientations of the forearms from the orientations of the hands (106, 108) and upper arms (105 and 103), e.g., using a technique disclosed in U.S. Pat. App. Ser. No. 15/787,555, filed Oct. 18, 2017 and entitled “Tracking Arm Movements to Generate Inputs for Computer Systems,” the entire disclosure of which is hereby incorporated herein by reference.

[0071] FIG. 2 illustrates a system to control computer operations according to one embodiment. For example, the system of FIG. 2 can be implemented via attaching the arm modules (113, 115) to the upper arms (105, 103) respectively, the head module (111 ) to the head (107) and/or hand modules (117, 119), in a way illustrated in FIG. 1.

[0072] In FIG. 2, the head module (111 ) and the arm module (113) have micro electromechanical system (MEMS) inertial measurement units (IMUs) (121 , 131 ) that measure motion parameters and determine orientations of the head (107) and the upper arm (103). [0073] Similarly, the hand modules (117, 119) can also have IMUs. In some applications, the hand modules (117, 119) measure the orientation of the hands (106, 108) and the movements of fingers are not separately tracked. In other applications, the hand modules (117, 119) have separate IMUs for the measurement of the orientations of the palms of the hands (106, 108), as well as the orientations of at least some phalange bones of at least some fingers on the hands (106, 108). Examples of hand modules can be found in U.S. Pat. App. Ser. No. 15/792,255, filed Oct. 24, 2017 and entitled “Tracking Finger Movements to Generate Inputs for Computer Systems,” the entire disclosure of which is hereby incorporated herein by reference.

[0074] Each of the IMUs (131 and 121 ) has a collection of sensor components that enable the determination of the movement, position and/or orientation of the respective IMU along a number of axes. Examples of the components are: a MEMS accelerometer that measures the projection of acceleration (the difference between the true acceleration of an object and the gravitational acceleration); a MEMS gyroscope that measures angular velocities; and a magnetometer that measures the magnitude and direction of a magnetic field at a certain point in space. In some embodiments, the IMUs use a combination of sensors in three and two axes (e.g., without a magnetometer).

[0075] The computing device (141 ) can have a prediction model (116) and a motion processor (145). The measurements of the IMUs (e.g., 131 , 121) from the head module (111 ), arm modules (e.g., 113, 115), and/or hand modules (e.g., 117, 119) are used in the prediction module (116) to generate predicted measurements of at least some of the parts that do not have attached sensor modules, such as the torso (101 ), and forearms (112, 114). The predicted measurements and/or the measurements of the IMUs (e.g., 131 , 121 ) are used in the motion processor (145).

[0076] The motion processor (145) has a skeleton model (143) of the user (e.g., illustrated FIG. 3). The motion processor (145) controls the movements of the parts of the skeleton model (143) according to the movements/orientations of the corresponding parts of the user. For example, the orientations of the hands (106, 108), the forearms (112, 114), the upper arms (103, 105), the torso (101 ), the head (107), as measured by the IMUs of the hand modules (117, 119), the arm modules (113, 115), the head module (111 ) sensor modules and/or predicted by the prediction model (116) based on the IMU measurements are used to set the orientations of the corresponding parts of the skeleton model (143).

[0077] Since the torso (101) does not have a separately attached sensor module, the movements/orientation of the torso (101) can be predicted using the prediction model (116) using the sensor measurements from sensor modules on a kinematic chain that includes the torso (101). For example, the prediction model (116) can be trained with the motion pattern of a kinematic chain that includes the head (107), the torso (101), and the upper arms (103, 105) and can be used to predict the orientation of the torso (101) based on the motion history of the head (107), the torso (101), and the upper arms (103, 105) and the current orientations of the head (107), and the upper arms (103, 105).

[0078] Similarly, since a forearm (112 or 114) does not have a separately attached sensor module, the movements/orientation of the forearm (112 or 114) can be predicted using the prediction model (116) using the sensor measurements from sensor modules on a kinematic chain that includes the forearm (112 or 114). For example, the prediction model (116) can be trained with the motion pattern of a kinematic chain that includes the hand (106), the forearm (114), and the upper arm (105) and can be used to predict the orientation of the forearm (114) based on the motion history of the hand (106), the forearm (114), the upper arm (105) and the current orientations of the hand (106), and the upper arm (105).

[0079] The skeleton model (143) is controlled by the motion processor (145) to generate inputs for an application (147) running in the computing device (141). For example, the skeleton model (143) can be used to control the movement of an avatar/model of the arms (112, 114, 105 and 103), the hands (106, 108), the head (107), and the torso (101 ) of the user of the computing device (141 ) in a video game, a virtual reality, a mixed reality, or augmented reality, etc.

[0080] Preferably, the arm module (113) has a microcontroller (139) to process the sensor signals from the IMU (131) of the arm module (113) and a communication module (133) to transmit the motion/orientation parameters of the arm module (113) to the computing device (141). Similarly, the head module (111) has a microcontroller (129) to process the sensor signals from the IMU (121 ) of the head module (111) and a communication module (123) to transmit the motion/orientation parameters of the head module (111 ) to the computing device (141 ).

[0081] Optionally, the arm module (113) and the head module (111 ) have LED indicators (137 and 127) respectively to indicate the operating status of the modules (113 and 111 ).

[0082] Optionally, the arm module (113) has a haptic actuator (138) respectively to provide haptic feedback to the user.

[0083] Optionally, the head module (111 ) has a display device (127) and/or buttons and other input devices (125), such as a touch sensor, a microphone, a camera (126), etc.

[0084] In some instances, a stereo camera (126) is used to capture stereo images of the sensor devices (113, 115, 117, 119) to calibrate their measurements relative to a common coordinate system, such as the coordinate system (100) defined in connection with a reference pose illustrated in FIG. 1. Further, the LED indicators (e.g., 137) of a sensor module (e.g., 113) can be turned on during the time of capturing the stereo images such that the orientation and/or identity of the sensor module (e.g. , 113) can be determined from the locations and/or patterns of the LED indicators.

[0085] When the LED lights are not captured in the images, or when the sensor device do not have LED lights, the orientation of the sensor module can be predicted based on an image of a portion of the user wearing the sensor device in a predefined manner. For example, an ANN can be used to determine the orientations of the wrist, palm, distal, middle and proximal phalanges of thumb and index finger from the image of the hand and forearm of the user; and the orientations can be further used in another ANN to determine the orientation of the sensor device.

[0086] In some implementations, the head module (111 ) is replaced with a module that is similar to the arm module (113) and that is attached to the head (107) via a strap or is secured to a head mount display device.

[0087] In some applications, the hand module (119) can be implemented with a module that is similar to the arm module (113) and attached to the hand via holding or via a strap. Optionally, the hand module (119) has buttons and other input devices, such as a touch sensor, a joystick, etc.

[0088] For example, the handheld modules disclosed in U.S. Pat. App. Ser. No. 15/792,255, filed Oct. 24, 2017 and entitled “Tracking Finger Movements to Generate Inputs for Computer Systems,” U.S. Pat. App. Ser. No. 15/787,555, filed Oct. 18, 2017 and entitled “Tracking Arm Movements to Generate Inputs for Computer Systems,” and/or U.S. Pat. No. 10,509,469 entitled “Devices for Controlling Computers based on Motions and Positions of Flands” can be used to implement the hand modules (117, 119), the entire disclosures of which applications are hereby incorporated herein by reference.

[0089] When a hand module (e.g., 117 or 119) tracks the orientations of the palm and a selected set of phalange bones, the motion pattern of a kinematic chain of the hand captured in the predictive mode (116) can be used in the prediction model (116) to predict the orientations of other phalange bones that do not wear sensor devices.

[0090] FIG. 2 shows a hand module (119) and an arm module (113) as examples. In general, an application for the tracking of the orientation of the torso (101 ) typically uses two arm modules (113, 115) as illustrated in FIG. 1. The head module (111 ) can be used optionally to further improve the tracking of the orientation of the torso (101 ). Fland modules (117, 119) can be further used to provide additional inputs and/or for the prediction/calculation of the orientations of the forearms (112, 114) of the user.

[0091] Typically, an IMU (e.g., 131 or 121 ) in a module (e.g., 113 or 111 ) generates acceleration data from accelerometers, angular velocity data from gyrometers/ gyroscopes, and/or orientation data from magnetometers. The microcontrollers (139 and 129) perform preprocessing tasks, such as filtering the sensor data (e.g., blocking sensors that are not used in a specific application), applying calibration data (e.g., to correct the average accumulated error computed by the computing device (141 )), transforming motion/position/orientation data in three axes into a quaternion, and packaging the preprocessed results into data packets (e.g., using a data compression technique) for transmitting to the host computing device (141 ) with a reduced bandwidth requirement and/or communication time.

[0092] Each of the microcontrollers (129, 139) may include a memory storing instructions controlling the operations of the respective microcontroller (129 or 139) to perform primary processing of the sensor data from the IMU (121 , 131 ) and control the operations of the communication module (123, 133), and/or other components, such as the LED indicators (137), the haptic actuator (138), buttons and other input devices (125), the display device (127), etc.

[0093] The computing device (141 ) may include one or more microprocessors and a memory storing instructions to implement the motion processor (145). The motion processor (145) may also be implemented via hardware, such as Application-Specific Integrated Circuit (ASIC) or Field-Programmable Gate Array (FPGA).

[0094] In some instances, one of the modules (111 , 113, 115, 117, and/or 119) is configured as a primary input device; and the other module is configured as a secondary input device that is connected to the computing device (141 ) via the primary input device. A secondary input device may use the microprocessor of its connected primary input device to perform some of the preprocessing tasks. A module that communicates directly to the computing device (141 ) is consider a primary input device, even when the module does not have a secondary input device that is connected to the computing device via the primary input device.

[0095] In some instances, the computing device (141 ) specifies the types of input data requested, and the conditions and/or frequency of the input data; and the modules (111 , 113, 115, 117, and/or 119) report the requested input data under the conditions and/or according to the frequency specified by the computing device (141 ). Different reporting frequencies can be specified for different types of input data (e.g., accelerometer measurements, gyroscope/gyrometer measurements, magnetometer measurements, position, orientation, velocity).

[0096] In general, the computing device (141 ) may be a data processing system, such as a mobile phone, a desktop computer, a laptop computer, a head mount virtual reality display, a personal medial player, a tablet computer, etc.

[0097] FIG. 3 illustrates a skeleton model that can be controlled by tracking user movements according to one embodiment. For example, the skeleton model of FIG. 3 can be used in the motion processor (145) of FIG. 2. [0098] The skeleton model illustrated in FIG. 3 includes a torso (232) and left and right upper arms (203 and 205) that can move relative to the torso (232) via the shoulder joints (234 and 241 ). The skeleton model may further include the forearms (215 and 233), hands (206 and 208), neck, head (207), legs and feet. In some instances, a hand (206) includes a palm connected to phalange bones (e.g., 245) of fingers, and metacarpal bones of thumbs via joints (e.g., 244).

[0099] The positions/orientations of the rigid parts of the skeleton model illustrated in FIG. 3 are controlled by the measured orientations of the corresponding parts of the user illustrated in FIG. 1. For example, the orientation of the head (207) of the skeleton model is configured according to the orientation of the head (107) of the user as measured using the head module (111 ); the orientation of the upper arm (205) of the skeleton model is configured according to the orientation of the upper arm (105) of the user as measured using the arm module (115); and the orientation of the hand (206) of the skeleton model is configured according to the orientation of the hand (106) of the user as measured using the hand module (117); etc.

[0100] For example, the tracking system as illustrated in FIG. 2 measures the orientations of the modules (111 , 113, ... , 119) using IMUs (e.g., 111 , 113, ... ). The inertial-based sensors offer good user experiences, have less restrictions on the use of the sensors, and can be implemented in a computational efficient way. Flowever, the inertial-based sensors may be less accurate than certain tracking methods in some situations, and can have drift errors and/or accumulated errors through time integration. Drift errors and/or accumulated errors can be considered as the change of the reference orientation used for the measurement from a known reference orientation to an unknown reference orientation. An update calibration can remove the drift errors and/or accumulated errors.

[0101] An optical tracking system can use one or more cameras (e.g., 126) to track the positions and/or orientations of optical markers (e.g., LED indicators, 137) that are in the fields of view of the cameras. When the optical markers are within the fields of view of the cameras, the images captured by the cameras can be used to compute the positions and/or orientations of optical markers and thus the orientations of parts that are marked using the optical markers. However, the optical tracking system may not be as user friendly as the inertial-based tracking system and can be more expensive to deploy. Further, when an optical marker is out of the fields of view of cameras, the positions and/or orientations of optical marker cannot be determined by the optical tracking system.

[0102] An ANN of the prediction model (116) can be trained to predict the measurements produced by the optical tracking system based on the measurements produced by the inertial-based tracking system. Thus, the drift errors and/or accumulated errors in inertial-based measurements can be reduced and/or suppressed, which reduces the need for re-calibration of the inertial-based tracking system. Further details on the use of the prediction model (116) can be found in U.S. Pat. App. Ser. No. 15/973,137, filedMay 7, 2018 and entitled “tracking User Movements to Control a Skeleton Model in a Computer System,” the entire disclosure of which application is hereby incorporated herein by reference.

[0103] Further, the orientations determined using images captured by the camera (126) can be used to calibrate the measurements of the sensor devices (111 , 113, 115, 117, 119) relative to a common coordinate system, such as the coordinate system (100) defined using a standardized reference pose illustrated in FIG. 1, as further discussed below.

[0104] FIG. 4 illustrates a technique to determine an orientation of a ring device for tracking user movements using an image (400) captured by a camera (126) on a head- mounted display (127) according to some embodiments. As illustrated in FIG. 4, a sensor device (401 ) is configured to have the form factor of a ring device suitable to be worn on a finger of the hand of the user. The sensor device (401 ) has an inertial measurement unit, similar to IMU (131 ) in an arm module (113). The sensor device (401 ) in the form of a ring can be worn on the middle phalange (403) of the index finger. The sensor device (401 ) is configured with a touch pad that can be ready touched by the thumb (405) to generate a touch input.

[0105] In some embodiments, the image (400) captured by the camera (126) is converted into the image in a black/white format for processing to recognize the orientations of predefined features. For example, the image (400) captured by the camera (126) can be processed by an ANN to determine the orientations of features, such as the forearm, wrist, palm, distal phalange of thumb, middle phalange of thumb, distal phalange of the index finger, middle phalange of the index finger, proximal of the index finger, and metacarpal of the index finger in palm connecting. Optionally, the system converts the original image (400) from a higher resolution into a lower resolution image in a black/white format to facilitate the recognize.

[0106] The orientations of the forearm, wrist, palm, distal phalanges, middle phalanges, and proximal phalanges and determined from the image of the hand and upper arm illustrated in FIG. 4, can be provided as input to an ANN to predict the orientation of the sensor device (401 ). The predicted orientation can be used to calibrate the orientation measurement generated by the inertial measurement unit configured in the sensor device (401 ). Further, the relative orientations of the sensor device (401 ) and the hand of the user can be used to determine the orientation of the hand based on the orientation measurement generated by the inertial measurement unit configured in the sensor device (401 ).

[0107] Further details and examples of the technique of orientation determination based on FIG. 4 can be found in U.S. Pat. App. Ser. No. 16/576,661 , filed Sep. 19, 2019, and entitled “Calibration of Inertial Measurement Units in Alignment with a Skeleton Model to Control a Computer System based on Determination of Orientation of an Inertial Measurement Unit from an Image of a Portion of a User,” the entire disclosure of which application is hereby incorporated herein by reference.

[0108] FIG. 5 shows a ring device (501 ) having components configured at locations according to some embodiments. For example, the ring device (501 ) of FIG. 5 can have an inertial measurement unit, similar to the sensor device (401 ) of FIG. 4.

[0109] The ring device (501 ) of FIG. 5 has a ring-shaped housing configured to be wrapped round a finger of a user when the device (501 ) is worn on a finger of the user (e.g., in a way as illustrated in FIG. 4). The overall shape of the ring device (501 ) of FIG. 5 is substantially circular. In other embodiments, the overall shape of the ring device (501 ) can have other shapes (e.g., elliptical, octagonal, rectangular, triangular, etc.). The ring device (501 ) of FIG. 5 can be in one of many form-factors and can be configured include components similar to the hand module (119) or the arm module (113) illustrated in FIG. 2 as a part of the tracking system. In some implementations, the ring device (501 ) of FIG. 5 can be used to replace a hand module (119) in the system of FIG. 2, where the orientation measurements from the sensor modules (e.g., 501 , 113, 111 ) are provided to a computing device (141 ) to control a skeleton model (143) of the user in a virtual reality application, a mixed reality application, an augmented reality application, etc. For example, the ring device (501 ) of FIG. 5 can provide orientation measurements to control the movement of parts of the skeleton model (143) of the user.

[0110] In FIG. 5, the ring device (501 ) can include a spring element (503) that extends from the housing of the ring device (501 ) for improved grip on the finger when the ring device (501 ) is worn on the finger.

[0111] The ring device (501 ) of FIG. 5 having an overall “c” shape with an opening (513). The gap between the two ends of the “c” shape at the opening (513) can be small, or substantially closed in other implementations.

[0112] FIG. 6 illustrates an alternative perspective view of the c-shaped ring device of FIG. 5. In other embodiments, an optional joint can be configured near the two ends to substantially hide the gap between the two ends of the c-shaped housing. For example, a joint (550) can be configured near the region (550) illustrated in FIG. 6. The housing of the ring device (501 ) of FIG. 5 (or FIG. 6) has structural discontinuity at the location near the opening (513).

[0113] At an opposite side (507) across the center (505) of the ring device (501) from the discontinuity point near the opening (513), the housing of the ring device (501 ) of FIG. 5 has a contiguous section (515).

[0114] In FIG. 5, the ring device (501 ) has a central axis X2 that is substantially in the direction of the lengthwise direction of a finger, when the ring device (501 ) is worn on the finger. The central axis X2 is at the center (505) of the ring device (501 ). The Y2 axis goes from the center (505) to the discontinuity point near the opening (513). The Z2 axis is perpendicular to the central axis X2 and the Y2 axis.

[0115] On the Y2 axis, the contiguous section (515) of the housing is at the opposite side of the center (505) relative to the discontinuity point at the opening (513).

[0116] In some embodiments, an antenna of the ring device (501 ) is configured in the contiguous section (515) at a location (507) that is on or near the Y2 axis. The location (507) is substantially at in the middle of the contiguous section (515) that extends from line YZ1 to line YZ2 in FIG. 5. Lines YZ1 and Z2 form angles that are about 45 degrees from axis Z2. Lines YZ2 and Y2 form angles that are about 25 degrees from axis Y2. Lines YZ1 and YZ2 form angles that are about 70 degrees. The contiguous section (515) between the lines YZ1 and YZ2 can be considered a left section (515) when the discontinuity point at the opening (513) is considered to be located to the right of the center (505).

[0117] The contiguous section (515) has an outer portion (517) and an inner portion (509) that is closer to the fwinger or the center (505) than the outer portion (507). The antenna can be attached to the outer portion (517) of the contiguous section (515) near the location (507) to reduce skin effects and/or proximity effects.

[0118] The antenna of the ring device (501 ) can be configured to communicate with the computing device (141 ) using a Bluetooth Low Energy technique (BLE). For example, the antenna configured at the location (507) can be a ceramic antenna, a printed circuit board antenna, a stamped metal antenna, or a wire antenna. For example, the ring device (501 ) can be configured to transmit, via the BLE antenna to the computing device (141 ), data indicating of orientation of the finger based on measurements of the inertial measurement unit of the ring device (501 ). For example, the inertial measurement unit can include a micro-electromechanical system (MEMS) gyroscope, a MEMS accelerometer and/or a magnetometer.

[0119] In general, the location (507) of the BLE antenna is preferably closer to the outer portion of the housing and further away from the inner portion of the housing to reduce skin-effects and proximity effects. For example, when the BLE antenna is configured to be located at the contiguous section (515) between the lines YZ1 and YZ2, the BLE antenna is most protected from external effects and can function properly without being disrupted by other electronic components of the ring device (501 ), such as a touch pad, a charging pad, etc.

[0120] Some components of the ring device (501 ) can contain metal elements. The BLE antenna is isolated from such components that may disrupt the operations of the BLE antenna. For example, the contiguous section (515) can be made of a dielectric material to isolate the BLE antenna (507) from such components that are configured outside of the section (515) between the lines YZ1 and YZ2. In some embodiments, the metal contained materials should be away from the BLE antenna at the distance which is no less than 3 mm. Therefore, the dielectric material can be smaller than the section (515) based on the location of the BLE antenna. The dielectric material can include one or more types of plastic such as Acrylonitrile Butadiene Styrene (ABS), Polypropylene (PP), Polycarbonates (PC), Polymethyl Methacrylate (PMMA), Polyethylene (PE), Polystyrene (PS), High Impact Polystyrene (HIPS), Thermoplastic Elastomer (TPE), Thermoplastic polyurethane (TPU), or Silicone, or any combination thereof.

[0121] Optionally, the ring device (501) has a light-emitting diode (LED) indicator configured at a location (511) that is on or near the Z2 axis and in an outer portion (531 ) of an upper contiguous section of the housing of the ring device (501 ). In some embodiments, the ring device (501) can have a light-emitting diode (LED) indicator configured at any portion (531, 533, 535)) of an upper contiguous section of the housing of the ring device (501 ).

[0122] In FIG. 5, the upper contiguous section (531 , 533, 535) is above the center (505) and is between the positive portion of axis Y2 and the line YZ1. The angle between the positive portion of axis Y2 and the line YZ1 is about 135 degrees in a cross-section view of the housing. For example, the upper contiguous section can include an outer portion (531), a middle portion (535), and an inner portion (533) that is closer to the center (505) than the middle portion (535) and the outer portion (531 ).

[0123] Optionally, the ring device (501) has a charging pad (533, 537, 539) configured to charge a battery configured in the ring device (501). For example, the charging pad (533, 537) can be configured in the upper part of the device at the inner portion (533, 537) of the upper contiguous section. Alternatively, the charging pad (537) can be configured in the lower contiguous section (539) that is between the positive portion of axis Y2 and the line YZ2. An angle between the positive portion of axis Y2 and the line YZ2 is no more than 155 degrees.

[0124] Optionally, the ring device (501 ) has a touch pad configured to receive touch input from a finger of the user. For example, the touch pad can be configured on the outer portion (531 ) of the upper contiguous section between the positive portion of axis Y2 and the positive portion of axis Z2. Thus, the touch pad extends no more than 90 degrees from the positive portion of axis Y2.

[0125] Optionally, the ring device (501 ) has an LED display configured to present output data to the user. For example, the LED display can be configured on the outer portion (531 ) of the upper contiguous section between the positive portion of axis Y2 and the positive portion of axis Z2. Thus, the LED display extends no more than 90 degrees from the positive portion of axis Y2.

[0126] Optionally, the ring device (501 ) has a fingerprint scanner configured to generate fingerprint data of the user of the ring device (501 ) to identify the user. For example, the fingerprint scanner can be configured on the outer portion (531 ) of the upper contiguous section between the positive portion of axis Y2 and the positive portion of axis Z2. Thus, the fingerprint scanner extends no more than 90 degrees from the positive portion of axis Y2

[0127] Optionally, the ring device (501 ) has a force sensor configured to detect the pressure of touch or click of the user while interacting with device. For example, the force sensor can be configured on the outer portion (531 ) of the upper contiguous section between the positive portion of axis Y2 and the positive portion of axis Z2. Thus, the force sensor extends no more than 90 degrees from the positive portion of axis Y2.

[0128] Optionally, the ring device (501 ) has a near field communication (NFC) marker configured to provide identification information through NFC communications. In some embodiments, the NFC maker can be configured in the lower contiguous section (539) that is between the positive portion of axis Y2 and the line YZ2. An angle between the positive portion of axis Y2 and the line YZ2 is no more than 135 degrees

[0129] Optionally, the ring device (501 ) has one or more biosensor components configured on an inner portion (533) of the upper contiguous section between the positive portion of axis Y2 and the line YZ1 . An angle between the positive portion of axis Y2 and the line YZ1 is no more than 155 degrees.

[0130] For example, the one or more biosensor components can include heart rate sensor (i.e. , optical heart rate sensor/photoplethysmography sensor) to measure heart rate in beats per minute or detect the pulse waves; piezoelectrical sensors to measure changes in pressure, acceleration, temperature, strain, or force; capacitive and/or optical sensors to detect if the user was wearing the device (500) and to detect and measure proximity of nearby objects; thermometer to measure temperature of the user; Manometer to measure blood pressure; galvanic skin sensor to measure skin resistance, skin conductance and stress level (i.e., sweating); electromyography sensor to measure the electrical activity of muscles; and/or CGM (continuous glucose monitoring) sensor to monitor glucose level.

[0131] Optionally, the ring device (501 ) has a microphone configured to collect voice inputs from a user for transmission to the computing device (141 ). In some embodiments, the microphone can be configured on the middle portion (535) of the upper contiguous section between the positive portion of axis Y2 and the line YZ1 . An angle between the positive portion of axis Y2 and the line YZ1 is no more than 135 degrees.

[0132] Optionally, the ring device (501 ) has a haptic actuator and/or speaker configured present the output feedback to its user. In some embodiments, the haptic actuator and/or speaker can be configured on the middle portion (535) of the upper contiguous section between the positive portion of axis Y2 and the line YZ1 . An angle between the positive portion of axis Y2 and the line YZ1 is no more than 135 degrees.

[0133] FIG. 7 and FIG. 8 illustrate an alternative ring device according to some embodiments of the disclosure. In FIG. 7, the ring device (700) is illustrated in an opened position while in FIG. 8, the ring device (700) is illustrated in a closed position.

[0134] In the illustrated embodiments, when in the closed position, the ring device (700) may be attached to a finger or phalange of a user, as described previously. Notably, the electronic components of the ring device (700) may be similar or identical to that of ring device (501 ) and details regarding the internal components of the ring device (700) that are similar or identical to ring device (501 ) are not repeated herein. For example, ring device (700) may include one or more IMUs to measure the position of the ring device (700) and may include one or more antennae (e.g., Bluetooth antennae) to transmit IMU data to a computing device (141 ) as described previously. The positioning and design considerations regarding the IMU and antennae described with respect to ring device (501 ) apply equally to the ring device (700). The ring device (700) may also include one or more LED elements to enable optical tracking, as described for ring device (501 ). In some embodiments, the ring device (700) may include a charging pad similar to charging pad (533, 537, 539). In some embodiments, the ring device (700) includes a touchpad similar to that of ring device (501 ). In some embodiments, the ring device (700) may include one or more of an LED display, fingerprint scanner, force sensor, NFC marker, biosensor components, microphone, or haptic actuator as described in connection with ring device (501 ). Details of these components are not repeated herein but are incorporated in their entirety. Various components may be described herein to the extent the placement of such components differs from the components described in connection with ring device (501 ). In the illustrated embodiment, electronic components of the ring device (700) may be situated in the upper portion (710) of the ring device (700).

[0135] In the illustrated embodiment, the ring device (700) includes a first L-shaped portion (702) and a second L-shaped portion (704). These portions (702, 704) are alternatively referred to as a base portion (702) and clip portion (704). In the illustrated embodiment, the clip portion (704) includes an end portion (706) designed to be inserted into a groove portion formed by two distal portions (708a, 708b) of the base portion (702). In the illustrated embodiment, the clip portion (704) rotates about axis (712) during opening and closing. In the illustrated embodiment, a user’s phalange is inserted in the opening (714) formed between clip portion (704) and base portion (702). In some embodiments, the ring device (700) may be used on a middle phalange of a user. However, the specific phalange is not limiting.

[0136] In some embodiments, the clip portion (704) includes a cylindrical cavity that allows for the insertion of a hinge rod therein. The ends of the hinge rod are then placed in corresponding receptacles of distal portions (708a, 708b). Thus, the hinge rod allows for rotational movement of the clip portion (704) with respect to the base portion (702). In another embodiment, the clip portion (704) includes two outward extrusions that are designed to be inserted into corresponding cavities on the inward sides of the distal portions (708a, 708b). In this embodiment, the clip portion (704) can “snap” into the base portion (702) and also be able to be rotationally moved with respect to the base portion (702).

[0137] In some embodiments, a spring element (illustrated in FIG. 10) is situated between the clip portion (704) and base portion (702). Thus, in some embodiments, the clip portion (704) is spring-loaded with respect to base portion (702). In one embodiment, a torsion spring is inserted near one distal portion (708a, 708b). In other embodiments, two torsion springs are inserted near both ends of the distal portions (708a, 708b). That is, a first spring is placed near distal portion (708a) and a second spring is placed near distal portion (708b). In one embodiment, the torsion springs used may one coil. In other embodiments, two or more coils may be used.

[0138] FIG. 8 illustrates the ring device (700) in a closed position. The embodiment illustrated in FIG. 7 may comprise the ring device (700) in a resting state. In this state, one end of the clip portion (704) contacts the inner surface of the body portion (702). In contrast, the embodiment illustrated in FIG. 8 comprises the ring device (700) in an opened state. In this state, a force may be exerted on the clip portion (704) and away from the body portion (702) by, for example, a user’s hand (not illustrated). In one embodiment, FIG. 8 illustrates the ring device (700) in a maximumly opened state. As described above, the use of a spring (e.g., a torsion spring) causes the clip portion (704) to return to the resting state illustrated in FIG. 7 when the force is removed.

[0139] FIG. 9 illustrates the ring device (700) in an opened and closed state. For the sake of clarity, clip portions (704a, 70b) comprise alternative state views of the same clip portion (704) as will be described herein.

[0140] In the opened state, the clip portion (704a) is separated from the body portion (702) resulting in a gap (904) wherein a user’s phalange can be inserted therein. In some embodiments, the illustrated clip portion (704a) is partially opened and can be extended further away from body portion (702) while a user inserts a phalange through gap (904) into gap (714). Due to the use of a spring element, when the user’s phalange fully passes through gap (904) into gap (714), the spring element causes the clip portion (704a) to move as indicated by the arrow to clip portion (704b).

[0141] FIG. 10 is a photograph illustrating a prototype of the ring device (700). In the illustrated embodiment, a user’s finger (1004) is inserted into gap (714) and clip portion (704) is moved inward toward body portion (702) by operation of a spring element (1002) surrounding hinge rod (902).

[0142] FIGS. 11-14 illustrate an alternative ring device (1100) employing a replicable “snail” connector mechanism. In the illustrated embodiment, the mechanism (1110) for attaching the ring device (1100) to a user’s finger or phalange with replaceable rubber parts (1104, 1106, 1108). The main body (1102) of the ring device (1100) comprises a universal part that can used with connector mechanisms (1110) having different sizes. As illustrated in FIG. 13 and FIG. 14, the mechanism (1110) can be fastened to main body portion (1102). In this case a rubber part of the mechanism (1110) can have the different sizes to fit any finger types. The size range can be reduced because of the snail-shaped form factor: the ring can change its size using the flexibility of the snail-shaped construction.

[0143] In the illustrated embodiment, a main body (1102) comprises an electronic device that may include various electronic components such as an IMU, LED, touchpad, etc. Details of the electronics included in main body (1102) have been described before, e.g., in the description of upper portion (710) of ring device (700) and are not repeated herein.

[0144] The main body (1102) connects to a connector mechanism (1110). In FIGS. 11 and 13, mechanism (1110) is depicted as having three portions (1104, 1106, 1108). However, in some embodiments, such as those depicted in FIGS. 12 and 14, the mechanism (1110) is a single structure and, thus, the individual portions (1104, 1106, 1108) may be included solely to explain various parts of the single mechanism (1110).

[0145] In the illustrated embodiments, the mechanism (1110) includes a connecting portion (1104). In the illustrated embodiment, the connecting portion (1104) is sized to match the bottom side of the main body (1102). In one embodiment, depicted in FIG. 14, the connecting portion (1104) includes two side walls and a cavity designed to receive the main body (1102). In the illustrated embodiment, the bottom side of the main body (1102) includes a thick groove (1402) and a rib (1404) on the distal portion of the bottom side. The combination of groove (1402) and rib (1404) forms a “track” that can attached to mechanism (1110). In one embodiment, the main body (1102) comprises a hard-plastic device housing the electronics and groove (1402) and rib (1404) comprise hard plastic. Although illustrated on one side, in the illustrated embodiments, the main body (1102) includes a groove and rib on each side of the device.

[0146] In the illustrated embodiments, the connecting portion (1104) includes a slot (1406) running along the circumference of the connecting portion (1104). The connecting portion (1104) also includes a projection (1408) running along the circumference of the connecting portion (1104). In the illustrated embodiment, the slot (1406) is configured to receive the rib (1404) and the projection (1408) is configured to be inserted into groove (1402). In the illustrated embodiment, the connecting portion (1104), and the mechanism (1110) comprise a pliable rubber. Thus, to connect the mechanism (1110) to the main body (1102) the connecting portion (1104) can be pressed against the lower surface of the main body (1102). During this operation, the projection (1408) expands over the rib (1404) and “snaps” into the groove (1402), locking the mechanism (1110) into place. Although illustrated as single-sided, the mechanism (1110) includes a corresponding slot and projection on both sides, corresponding to the main body (1102).

[0147] As illustrated, in some embodiments, the groove (1402), rib (1404), slot (1406), and projection (1408) are tapered. Specifically, in the illustrated embodiments, the portions (1402-1408) increase in width while traveling along the circumference of the main body (1102) and mechanism (1110). In some embodiments, this tapering aids in positioning the main body (1102) on the mechanism (1110) and aligns the components before connecting.

[0148] In the illustrated embodiment, the connecting portion (1104) is sized to match the main body (1102) and thus may be the same size regardless of the other portions (1106, 1108). In contrast, the two portions (1106, 1108) may vary in size based on the desired size of gap (1112). Thus, portions (1106, 1108) may be varied in size, varying the size of gap (1112). In this manner, the size of gap (1112) can be adjusted to more finely matched the size of a user’s phalange or finger.

[0149] In some embodiments, both the holder portion (1108) and lock portion (1106) may comprise semi-rigid plastic that is bendable and posable by a user. The portions (1106, 1108) may be made of a material that can be bent and “fixed” in position. In this embodiment, the lock portion (1106) can be configured to “wrap” around a finger or phalange of a user to attached device (1100) to the user. In some embodiments, the user can insert a finger into gap (1112) while the lock portion (1106) continues to exert pressure inward on the finger, forming a lock on the user’s finger.

[0150] FIG. 15 is a diagram of an alternative ring design according to some embodiments of the disclosure. It is noted that the dimensions depicted in FIG. 15 are exemplary only and should not be construed as limiting the disclosure.

[0151] Returning to ring device (700) described in FIGS. 7-10, a cross-sectional view of the opening (714) formed by base portion (702) and clip portion (704) is depicted.

[0152] Notably, when in the closed position, the bottom sides (1502, 1504) of the base portion (702) and clip portion (704) are substantially oval in shape. In contrast, when using the ring with common circle shape (as depicted, for example, in FIG. 5), the ring can be rotated around the finger phalange (1506). It may adversely affect the user experience (e.g., a user can’t place the touchpad surface correctly) and can cause additional errors during calculation processes (when position/orientation of the ring device is calculated).

[0153] By contrast, reduce or eliminate such rotational errors, the oval shape formed by sides (1502, 1504) may be used. As illustrated, the inner surface of the ring device (700) includes two substantially flat portions (1502) and two substantially circular portions (1504). The flat portions (1502) actively prevent the ring device (700) from rotating about the user’s finger or phalange (1506) and thus prevent slippage of the ring device (700).

[0154] FIGS. 16-23 are diagrams of an armband device and mount according to some embodiments of the disclosure.

[0155] In the illustrated embodiment, a sensor (1602) is placed within a retainer (1604) via a receptacle (1608). In one embodiment, the sensor (1602) comprises an arm module (113, 115) as described previously. Details of the arm modules (113, 115) are not repeated herein. In brief, the sensor (1602) may include various electronic components for measuring the position of the sensor (e.g., accelerometer, IMU, etc.) as well as various input (e.g., touchpad) and output (e.g., audio, haptic, etc.) components. In the illustrated embodiment, the sensor (1602) includes an LED component (1610) for performing optical tracking and/or providing visual feedback (e.g., that the sensor is powered, charging, etc.). However, the specifically labeled component (1610) should not be construed as limiting the components of the sensor (1602), including components embedded within the sensor (1602) and thus not visible from the exterior of the component. Details of the sensor (1602) will be provided in more detail herein.

[0156] As illustrated in FIG. 18, the sensor (1602) is detachable from the retainer (1604). Retainer (1604) includes a receptacle portion (1608) that is shaped according to the shape of the sensor (1602). In the illustrated embodiment of FIG. 18, the receptacle (1608) includes a pass-through portion (1802). This portion (1802) is designed to allow a haptic portion (1702) of the sensor (1602) to pass through the retainer (1604) unimpeded. In one embodiment, haptic portion (1702) comprises a haptic output interface, as will be described in more detail herein. In the illustrated embodiment of FIG. 18, the receptacle (1608) includes a plurality of magnetic elements such as element (1706a). In some embodiments, the sensor (1602) includes matching magnetic elements on the underside of the sensor (1602). These matching magnetic elements allow the sensor (1602) to be magnetically connected to the retainer (1604), avoiding the use of clips, snaps, or other connection elements. In one embodiment illustrated in FIG. 17, the receptacle (1608) and sensor (1602) includes three magnetic elements (1706a-c, 2002a-c). As illustrated in FIG. 20, in some embodiments, magnetic elements on the sensor (1602) may be visible (2002a, 2002b) while other magnetic elements (2002c) are not visible on the exterior of the sensor (1602).

[0157] As illustrated in FIGS. 18-23, the sensor includes a haptic portion (1702). This haptic portion (1702) extends away from the top surface (1902) of the sensor. In some embodiments, the haptic portion (1702) includes a haptic output interface that provides haptic feedback to users. For example, the haptic interface can provide physical vibrations. Other types of physical sensors may be used. Additionally, the sensor (1602) may include a touch pad or other input device on the surface (1902) of the sensor.

[0158] In one embodiment, the sensor (1602) can be magnetically attached to the retainer (1604). Straps (not illustrated) may then be inserted in strap slots (1606a, 1606b). By using these straps, the sensor (1602) and retainer (1604) can be physically attached to a user’s arm or other limb, as described previously. Thus, in this configuration, the sensor (1602) acts as a limb sensor (e.g., a forearm sensor). Notably, since the retainer (1604) includes an opening (1802) and the sensor (1602) includes a haptic portion (1702), the sensor (1602) can directly touch a user’s skin when attached to the retainer (1604) and, ultimately, attached to a user. FIGS. 22-23 provide detail on the receptacle (1608), excluding the planar portions of the retainer (1604) including the slots (1606a, 1606b). As can be seen in FIG. 23, the haptic portion (1702) protrudes slightly from the lower surface of the receptacle (1608). In one embodiment, the haptic portion (1702) extends 0.4 mm from the surface of the receptacle and, by proxy, the retainer (1604). The specific protrusion distance may be configured based on needs of the system. This slight protrusion allows for direct skin contact of the haptic portion (1702) when strapped to a user’s limb, thus increasing the sensation provided by a haptic output device.

[0159] In another embodiment, the sensor (1602) can be physically removed from the retainer (1604), as illustrated in FIG. 18. As one example, while the retainer (1605) is strapped to a user’s arm, the user can manually remove the sensor (1602) from the retainer (1604) by pulling on the sensor (1602) and thus disengaging the magnets (2002a-2002c) from the corresponding magnets on the receptacle (1706a-1706c). FIGS. 19-21 illustrate the sensor in a “stand alone” mode. As discussed, the sensor (1602) may include a touchpad or other input device (1902). As one example, the sensor (1602) may be configured to be gripped such that a user’s thumb is positioned above LED (1610) and the portion of the touch pad (1902) above LED (1610) may comprise a manual clickable input device.

[0160] Thus, when removed from retainer (1604), the sensor (1602) may be operable as an independent input device. In some embodiments, in a stand-alone mode, the sensor (1602) can be used as a clicker, remote, two-dimensional touchpad/mouse, fitness tracker, three degrees of freedom (3DoF) input device or other type of input device. Since the sensor (1602) includes communications circuity (e.g., a Bluetooth transceiver), the sensor (1602) can receive input events and transmit these input events to a data processing system, as described previously. Finally, the sensor (1602) additionally includes a charging element (2004). This element allows the sensor (1602) to be charged by a charging station. Details of charging the sensor (1602), and ring devices (700, 1100) are described in more detail herein.

[0161] FIG. 24 is a perspective view of a charging case for an armband module and/or ring device according to some embodiments of the disclosure.

[0162] In the illustrated embodiment, a charging case (2400) is depicted which can simultaneously charge one or more of a ring device (2404) or armband module (2402). In the illustrated embodiment, the ring device (2404) corresponds to the ring devices (500, 700, 1100) described previously while armband module (2402) corresponds to the sensor (1602) described previously. As illustrated, the case (2400) includes two receptacles to house the devices (2402, 2404). The devices (2402, 2404) include charging elements that enable electrical connections between the devices (2402, 2404) and a circuit board within the base (2408) of the case (2400). The case (2400) additionally includes a lid portion (2406) that is connected to the base (2408) via a hinge or similar element. The lid portion (2406) is designed to close on top of the receptacles and devices (2402, 2404). In some embodiments, magnetic elements may be placed opposite the hinge side of the case (2400) to allow the lid to “snap” shut when closed as well as retain the lid (2406) when connected to the base (2408).

[0163] In the illustrated embodiment, the base (2408) includes a charging port (2410) and, optionally, one or more LED indicators (2412). In one embodiment, the port (2410) may comprise a USB-C port or similar port to charge an internal battery (not illustrated) of the case (2400). As discussed, an internal printed circuit board (PCB) may be inserted in the base (2408). The PCB may be connected to an internal, re-chargeable battery (e.g., a Lithium-Ion battery) and includes at least one power management integrated circuit (PMIC) or similar circuitry to recharge the battery when external power is connected to the PCB via charging port (2410). Additionally, the PCB may drive one or more LED indicators (2412) which provide a visual indication of the status of the device. In the example of four indicators, all indicators off may indicate no charge of the battery while one, two, three or four lit indicators may indicate increasing levels of charge. Certainly, other arrangements of visual indications may be used. [0164] Notably, in the illustrated embodiment, devices (2402, 2404) are interchangeable and thus can be swapped during charging. Details of the power connections enabling this interchangeable configuration are provided in the description of FIG. 31.

[0165] FIG. 25 is a perspective view of an alternative charging case for an armband module and/or ring device according to some embodiments of the disclosure. FIG. 26 is top perspective view of the alternative charging case according to some embodiments of the disclosure. FIG. 27 is a side view of the alternative charging case according to some embodiments of the disclosure. FIG. 28 is a cutaway view of the alternative charging case illustrating the internal circuitry of the case according to some embodiments of the disclosure. FIG. 29 is a front-bottom perspective view of the alternative charging case according to some embodiments of the disclosure. FIG. 30 is a bottom view of the alternative charging case according to some embodiments of the disclosure.

[0166] Various details of the alternative case (2500) are similar to that of case (2400) and those details are not repeated. Similar to case (2400), the case (2500) includes a lid (2504) and base (2506) that are connected via a hinge element (2702). As in case (2400), magnetic elements may be positioned on the lid (2504) (and corresponding position on the base, 2506) opposite the hinge to “snap” the lid (2504) close. The base (2506) also includes LED indicators (2510) which may operate similarly to LED indicators (2412). As indicated in FIGS. 29-30, the base (2506) may include a charging port (2902) such as a USB-C charging port. In one embodiment, the port (2902) may comprise a USB-C port or similar port to charge an internal battery (not illustrated) of the case (2500). As discussed, an internal printed circuit board (PCB) may be inserted in the base (2506). The PCB may be connected to an internal, re-chargeable battery (e.g., a Lithium-Ion battery) and includes at least one power management integrated circuit (PMIC) or similar circuitry to recharge the battery when external power is connected to the PCB via charging port (2902).

[0167] In addition to the foregoing, the case (2500) may include a divot (2508) opposite the hinge (2702) that enables a user to easily open the case (2500). As discussed, magnetized elements may be placed on either portion forming the divot to enable “snapping” the lid (2504) to the base (2506). [0168] In the illustrated embodiment, the lid (2504) includes a plurality of receptacles (2602, 2604). In the illustrated embodiment, these receptacles (2602, 2604) are designed to surround the exposed portions of devices (2402, 2404) and secure the devices (2402, 2404) therein the case. In the illustrated embodiment, the receptacles (2602, 2604) are sized such that they can receive either device (2402, 2404). Details of the receptacles (2602, 2604) are provided in FIGS. 31 and 32.

[0169] FIG. 28 illustrates the internal circuitry of the case (2500). Similar circuitry may be present in case (2400). As illustrated, the base of the case (2500) includes a PCB (2802) which may include various components (not illustrated) for managing power distribution from an external power source (not illustrated) and to an internal battery (not fully illustrated). Additionally, the components of the PCB (2802) manage power distribution and monitoring to devices (2402, 2404) as well as the driving of LED indicators (2804).

[0170] As illustrated for device (2404), cathode (2806a) and anode (2806b) elements extend from the PCB (2802) and are electrically coupled to corresponding pads of device (2404). In some embodiments, these connectors (2806a, 2806b) and pads comprise copper or similar conductive elements. These elements are designed to recharge the internal battery of the device (2404) when in contact. The PCB (2802) additionally has a magnetic connector (2804) designed to secure the device (2404) to the case and charging elements (2806a, 2806b). Although not visible, additional magnetic elements or charging elements may be present as depicted, for example, in FIG. 20

[0171] Although not visible, an identical (but mirror) configured of magnetic elements (2804) and electrically conductive elements (2806a, 2806b) may be present to hold and charge device (2402).

[0172] FIG. 31 is a cut away view of the alternative case according to some embodiments of the disclosure. FIG. 32 is a perspective view of a lid portion of the alternative case according to some embodiments of the disclosure.

[0173] In FIG. 31 , the same cavity (3102) is illustrated when holding devices (2402, 2404). In a first configuration (3100a), the cavity (3102) holds a ring device (2404). In a second configuration (3100b), the same cavity (3102) can hold an armband module (2402). The cavity (3102) is sized such that both devices (2402, 2404) can be arbitrarily placed in the case (2500). As described in FIG. 28, the devices (2402, 2404) are connected to PCB (2802) via magnetic (2804) and electrically conductive elements (2806b).

[0174] FIG. 32 provides an external perspective view of the cavity (3102). In the illustrated embodiment, two cavities (3204a, 3204b) are present in the base portion (2506) and two corresponding cavities (2602, 2604) are present in the lid portion (2504). When closed, these corresponding cavities (e.g., 2602, 3204a) form a cavity similar to cavity (3102) depicted in FIG. 31. Further, in FIG. 32, conductive elements (3204) and magnetic elements (3202a, 3202b) are depicted on distal ends of the cavities in contrast to central to the base (2506) as depicted in FIG. 28. Further, in FIG. 32, four conductive elements (3204) (e.g., cathode/anodes) are depicted. Nothing the specification should be construed as limiting the disclosure to a specific configuration.

[0175] The present disclosure includes methods and apparatuses which perform these methods, including data processing systems which perform these methods, and computer readable media containing instructions which when executed on data processing systems cause the systems to perform these methods.

[0176] For example, the computing device (141 ), the arm modules (113, 115) and/or the head module (111 ) can be implemented using one or more data processing systems.

[0177] A typical data processing system may include an inter-connect (e.g., bus and system core logic), which interconnects a microprocessor(s) and memory. The microprocessor is typically coupled to cache memory.

[0178] The inter-connect interconnects the microprocessor(s) and the memory together and also interconnects them to input/output (I/O) device(s) via I/O controller(s). I/O devices may include a display device and/or peripheral devices, such as mice, keyboards, modems, network interfaces, printers, scanners, video cameras and other devices known in the art. In one embodiment, when the data processing system is a server system, some of the I/O devices, such as printers, scanners, mice, and/or keyboards, are optional. [0179] The inter-connect can include one or more buses connected to one another through various bridges, controllers and/or adapters. In one embodiment the I/O controllers include a USB (Universal Serial Bus) adapter for controlling USB peripherals, and/or an IEEE-1394 bus adapter for controlling IEEE-1394 peripherals.

[0180] The memory may include one or more of: ROM (Read Only Memory), volatile RAM (Random Access Memory), and non-volatile memory, such as hard drive, flash memory, etc.

[0181] Volatile RAM is typically implemented as dynamic RAM (DRAM) which requires power continually in order to refresh or maintain the data in the memory. Non volatile memory is typically a magnetic hard drive, a magnetic optical drive, an optical drive (e.g., a DVD RAM), or other type of memory system which maintains data even after power is removed from the system. The non-volatile memory may also be a random access memory.

[0182] The non-volatile memory can be a local device coupled directly to the rest of the components in the data processing system. A non-volatile memory that is remote from the system, such as a network storage device coupled to the data processing system through a network interface such as a modem or Ethernet interface, can also be used.

[0183] In the present disclosure, some functions and operations are described as being performed by or caused by software code to simplify description. However, such expressions are also used to specify that the functions result from execution of the code/instructions by a processor, such as a microprocessor.

[0184] Alternatively, or in combination, the functions and operations as described here can be implemented using special purpose circuitry, with or without software instructions, such as using Application-Specific Integrated Circuit (ASIC) or Field- Programmable Gate Array (FPGA). Embodiments can be implemented using hardwired circuitry without software instructions, or in combination with software instructions. Thus, the techniques are limited neither to any specific combination of hardware circuitry and software, nor to any particular source for the instructions executed by the data processing system. [0185] While one embodiment can be implemented in fully functioning computers and computer systems, various embodiments are capable of being distributed as a computing product in a variety of forms and are capable of being applied regardless of the particular type of machine or computer-readable media used to actually effect the distribution.

[0186] At least some aspects disclosed can be embodied, at least in part, in software. That is, the techniques may be carried out in a computer system or other data processing system in response to its processor, such as a microprocessor, executing sequences of instructions contained in a memory, such as ROM, volatile RAM, non-volatile memory, cache or a remote storage device.

[0187] Routines executed to implement the embodiments may be implemented as part of an operating system or a specific application, component, program, object, module or sequence of instructions referred to as “computer programs.” The computer programs typically include one or more instructions set at various times in various memory and storage devices in a computer, and that, when read and executed by one or more processors in a computer, cause the computer to perform operations necessary to execute elements involving the various aspects.

[0188] A machine readable medium can be used to store software and data which when executed by a data processing system causes the system to perform various methods. The executable software and data may be stored in various places including for example ROM, volatile RAM, non-volatile memory and/or cache. Portions of this software and/or data may be stored in any one of these storage devices. Further, the data and instructions can be obtained from centralized servers or peer to peer networks. Different portions of the data and instructions can be obtained from different centralized servers and/or peer to peer networks at different times and in different communication sessions or in a same communication session. The data and instructions can be obtained in entirety prior to the execution of the applications. Alternatively, portions of the data and instructions can be obtained dynamically, just in time, when needed for execution. Thus, it is not required that the data and instructions be on a machine readable medium in entirety at a particular instance of time.

[0189] Examples of computer-readable media include but are not limited to non- transitory, recordable and non-recordable type media such as volatile and non-volatile memory devices, read only memory (ROM), random access memory (RAM), flash memory devices, floppy and other removable disks, magnetic disk storage media, optical storage media (e.g., Compact Disk Read-Only Memory (CD ROM), Digital Versatile Disks (DVDs), etc.), among others. The computer-readable media may store the instructions.

[0190] The instructions may also be embodied in digital and analog communication links for electrical, optical, acoustical or other forms of propagated signals, such as carrier waves, infrared signals, digital signals, etc. However, propagated signals, such as carrier waves, infrared signals, digital signals, etc. are not tangible machine readable medium and are not configured to store instructions.

[0191] In general, a machine readable medium includes any mechanism that provides (i.e., stores and/or transmits) information in a form accessible by a machine (e.g., a computer, network device, personal digital assistant, manufacturing tool, any device with a set of one or more processors, etc.).

[0192] In various embodiments, hardwired circuitry may be used in combination with software instructions to implement the techniques. Thus, the techniques are neither limited to any specific combination of hardware circuitry and software nor to any particular source for the instructions executed by the data processing system.

[0193] In the foregoing specification, the disclosure has been described with reference to specific exemplary embodiments thereof. It will be evident that various modifications may be made thereto without departing from the broader spirit and scope as set forth in the following claims. The specification and drawings are, accordingly, to be regarded in an illustrative sense rather than a restrictive sense.