CROSS REFERENCE TO RELATED APPLICATIONS [0001] Applicant claims priority to Vietnamese Application No. 1-2020-03551, filed on

18 June 2020, the disclosure of which is incorporated herein by reference.

TECHNICAL FIELD OF THE INVENTION [0002] The present invention relates to the field of automation entertainment robot, more particularly, to a robotic arm, a robotic hand, and a robotic system for use in playing the piano.

BACKGROUND OF THE INVENTION

[0003] The development of robots has opened various applications in industries, including entertainment robots, more particularly, piano-playing robots.

[0004] In the prior art, automatic piano players have been developed for a long time. Initial forms were rather simple since a solenoid is mounted to each piano key indirectly. Automatic system for playing the piano can replay exactly a song played on the piano by equipping a sensing system. Some solutions were used aiming to teach people to play piano. In addition to the automatic playing systems that there is a mechanism as equipped above, the ones with actuated means are placed outside robot have been presented. Some systems were developed with humanoid robot form with robotic hands. However, those systems are firstly bulky with huge size since they implemented a big number of solenoids at once as actuated means for pressing piano keys. If the number of solenoids is not sufficient, systems for moving the hand is bulky and not suitable for commercialization. With a big number of solenoids and huge hand moving mechanisms, previous player systems were expensive and unfriendly, hard to reach to popular users. [0005JUS7891935B2 disclosed a dual arm robot wherein each arm has a set of link/joints linked with an actuator and defines at least three degrees of freedom to generate independent horizontal translation and rotational of the last link. Besides, mechanism for vertical motion is also presented. However, this system is used in wafer transferring system so, an additional actuator is needed for directional control.

[0006] US5064340B2 disclosed an arm structure composed of two or three links pivotally connected to one another with a hand or end effector at the distal end of the distal link. Straight line movement of the pivoting mounting place of the end effector is provided from a rotating drive wheel coaxial with the pivoting of the proximal end of the proximal link. A drive wheel coaxially with the pivot of the proximal end of the proximal link causes the first and the second links to pivot about their pivot axis. Q and Z motors also are present. Z translation of this disclosure is up/down motion requiring additional motor. This disclosure uses one more additional motor coupled with robot base to control the direction of the arm (i.e., control the direction of the hand). Also, the mechanism in the disclosure is bulky and sophisticated.

[0007] The inventor finds that previous systems are often bulky, expensive, and not in robot- forms linear moving systems were preferably equipped to move the robot’s hand, which is huge and not friendly. Especially, it does not allow users to play the piano instantly without moving the robots. Previous hand designs tend to use a single solenoid for pressing a piano key, so the hand is quite big, or the mechanism for alternating the fingers is often bulky which is not suitable for commercialization. Huge size and heavy weight of previous hand designs also do not allow the robots to play songs with high tempo which requires agile movements.

[0008] Thus, there is a need to provide a robot that can play music with complicated chords with a high ability of commercialization and cost-reduction. SUMMARY OF THE INVENTION

[0009] The present invention aims to overcome technical disadvantages of prior art. In order to achieve the object, the invention provides a robotic arm, a robotic hand, and a robotic system for playing the piano. The robot arm uses only one minimum actuator to control motions of the hand. In addition, the design of the robotic hand allows the robot to play various complicated chords, but it is also light and compact so the robot can play songs with high tempo. Furthermore, the robotic system can be folded when it is not in use, so it does not interfere to users when the users want to play the piano. The design of the whole system is compact and friendly with users.

[0010] According to the first aspect, the present invention proposes a robotic arm for playing the piano comprises a first link, a second link, and a robot hand, in which the said first link has the same length as that of the said second link, and a distal end of said first link is pivotally connected to a proximal end of the second link; the arm further comprises: a base, an actuator is mounted to proximal end of the said first link, a controller, a first transmission system comprises: a first pulley mounted coaxially to the actuator by a shaft at proximal end of said first link, to drive the said second pulley through a first transmission means, a second pulley mounted at distal end of the said first link, in the same side with said first pulley, wherein a transmission ratio of said first transmission system is 1 :2, and a second transmission system comprises: a third pulley mounted to proximal end of said second link, that is coaxial with the said second pulley, and placed at the other side of the proximal end of the said second link, to drive the said fourth pulley through a second transmission means, a fourth pulley placed at distal end of the said second link and coupled with the said robot hand, wherein a transmission ratio of said second transmission system is 2: 1.

[0011] In another embodiment of present invention, the robot arm further comprises a third transmission system, comprising: a fifth pulley mounted to the actuator to drive a sixth pulley through a third transmission means, a sixth pulley mounted to the first link at an end of the base, and placed at the same side as the actuator, the sixth pulley mounted to be coaxial with the said first pulley,

[0012] According to the second aspect, the present invention provides a robotic hand coupled to the robot arm used to play the piano, wherein the hand comprises a first block, a second block, and a third block, and a plurality of fingers in the hand, wherein: said first block is provided between said second block, and said third block, wherein said second block and said third block are configured to be horizontally moved in respect to said first block, a plurality of fingers is rotatably provided in respect to said first block, said second block, and said third block, wherein the plurality of fingers has different dimensions, wherein: a plurality of longer fingers has a length sufficient to touch black keys of the piano; a plurality of shorter fingers has a length to touch white keys of the piano, a means for controlling operations of fingers is provided to be mounted on each of said first block, said second block, and said third block, each of fingers is controlled by the said means for controlling operations of fingers, wherein said first block comprises a plurality of bearings holding means that is mounted to said first block, each of a plurality of bearings holding means comprises a plurality of linear bearings, and is equipped with a position control means to control the positions of each of said second block and said third block in respect to said first block, wherein each of said second block, and said third block is provided with a plurality of sliding means, which are mounted to each of said second block, and said third block and are slidable with said linear bearings, and wherein each of said second block and said third block is respectively mounted to outputs of said position control means of each of said second block and said third block in a way that each of said blocks is horizontally travelable to in respect to said first block.

[0013] According to the third aspect, the present invention provides a robotic system for playing the piano comprising at least two robotic arms according to the present invention. [0014] According to the fourth aspect, the present invention provides a computer readable storage medium comprises at least one controller to implement instructions on which, to control the robotic system in accordance with the present invention.

[0015] This summary is provided to introduce simplified concepts concerning a robotic arm, a robotic hand, and a robotic system for playing the piano, which is further described below in the Detailed Description and Drawings. This summary is not intended to identify essential features of the claimed subject matter, nor is it intended for use in determining the scope of the claimed subject matter. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Detailed description of one or more aspects of the present invention is explained by reference to the following figures. The same reference numbers are used throughout the drawings to denote the same technical features and parts and the like: Fig. 1 shows an illustrated piano playing robotic system in accordance with an embodiment of the present invention.

Fig. 2 shows a robot arm moving along the piano in accordance with \ the present invention.

Fig. 3 shows a working principle of a robotic arm in accordance with the present invention. Fig. 4A shows a configuration of a robotic arm in accordance with an embodiment of the present invention.

Fig. 4B shows a configuration of a robot arm in accordance with another embodiment of the present invention. Fig. 5 shows a robotic hand structure in accordance with an embodiment of the present invention.

Fig. 6 shows dispositions of the first block and the second block of the robotic hand in separate state in accordance with an embodiment of the present invention.

Fig. 7 shows a configuration of an example robot hand in accordance with an embodiment of the present invention.

Fig. 8 shows a configuration of an example robot hand, illustrating the movement direction of the second block in accordance with an embodiment of the present invention.

Fig.9A, Fig.9B, Fig.9C, Fig.9D illustrate a robotic hand showing moving blocks in states placed on the piano.

Fig. 10 shows a robotic finger structure in accordance with an embodiment of the present invention.

Fig. 11 shows a dual arm robot system in accordance with an embodiment of the present invention. Fig. 12 shows a dual arm robot system mounted to a support frame in accordance with an embodiment of the present invention.

Fig. 13 shows an illustrated support frame in accordance the present invention.

Fig.l4A, Fig.l4B, Fig.l4C, Fig.l4D, and Fig.l4E illustrate a dual arm robot system in difference modes (playing piano, and steps to fold robot when it is not in use) in accordance with the present invention.

Fig. 15 shows a robotic control system in accordance with an embodiment of the present invention.

Fig. 16 is a flowchart showing a method for controlling a piano playing robotic system in accordance with an embodiment of the present invention. Fig. 17 is a flowchart showing a method for controlling a piano playing robotic system in accordance with an embodiment of the present invention. Fig. 18 shows a flowchart of an example method for controlling in accordance with the present invention.

DETAILED DESCRIPTION OF THE INVENTION

[0017] Herein terms are used to describe the present invention and its embodiments, which not aim for limiting the scope of the invention.

[0018] As used herein, a robotic hand, a robotic arm, and a robotic system for playing the piano according to the present invention also can be used to play other similar instruments, in example, but not limited to, electronic organs.

[0019] As used herein, the piano may relate to an electrical piano, an acoustic piano, or the like.

[0020] As used herein, black, and white keys relate to acoustic piano keys which are well-known by an ordinary skilled person in the art. Specifically, the white keys may present seven basic musical notes (A, B, C, D, E, F and G). The black keys may present seven sharp notes corresponding to seven basic musical notes (which raises a note by semitone) or may present seven flat notes (which lowers a note by a semitone).

[0021] As used herein, the terms “upper”, “lower”,” right”, “left”, “front”, “behind”, “above”, “below”, “outside”, “inside” and the terms related to locations such as “vertical”, “horizontal” are based on their relative locations illustrated in the figures and do not limit the scope of the invention in case directions of views are changed.

[0022] As used herein, the terms “proximal end” and “distal end” relate to the positions of shoulder, elbow, and wrist joints. Specifically, in a preferred embodiment, a proximal end of the first link may be on the shoulder on the first link, a distal end may be on the elbow of the first link. Similarly, in a preferred embodiment, a proximal end of second may be on the elbow of the second link while a distal end can be on wrist of the second link.

[0023] As used herein, throughout the specification, an example transmission system using the pulley in accordance with the present invention is the one including pulleys and power transmission means, which fit together. According to the present invention, they can be a belt pulley transmission system, for example, such as, any timing belt pulley transmission system, which is not limited by designs, materials, dimensions, or the number of teeth.

[0024] As used herein, a solenoid may relate to a typical electromagnetic solenoid which is well-known by a skilled ordinary person in the art. For example, typical electromagnetic solenoids may comprise, but not limited to, a coil placed inside a metal housing, a metal shaft slidable inside the coil. When a current is supplied, an electromagnetic force is created to attract the metal shaft moved inside. When the current is removed, the plunger is returned to its home position. An additional control driver can be used to control the working of the solenoid.

Overview of the robotic system

As shown in Fig.l, Fig. 1 illustrates a robotic system in accordance with the present invention is composed of a right arm 14R and a left arm 14L wherein the right arm and the left arm are coupled to a base 21, the right hand 15R and the left hand 15Lto play a piano 16 are coupled with the respective arms 14R and 14L. The right hand 15R and the left hand 15L are provided with multiple fingers. The whole robotic system is configured to be controlled by a controller 12. Users may select a song for robot to play by using a controlling device 11, in example, a smart phone, a tablet or any suitable device. The controller 12 is configured to be able to receive control commands from the controlling device 11, to control the two arms 14R, 14L to move the two hands 15R, 15L to desired positions to play the song. [0025] The robotic system is controlled by a controller 12 which can be placed directly inside the base 21. Preferably, the controller 12 is only in charge of controlling the motions of the robot but it is not necessary for the controller 12 to deal with algorithms used to work with musical data. When a user selects a ready song which has been processed by algorithms to provide operations of the robot, the robot would simply implement the required operations to play the music. With a new song (with music format, such as MIDI), which is not ready in the algorithms-processed list, the MIDI music file will be sent via internet to a server that that can process the musical algorithms for processing MIDI file. The result, which is a file of robot operations, will be sent back to robot the user ’ s smartphone/tablet. In another embodiment of the invention, the music algorithms may be implemented directly in the smartphone, the tablet, or the controller without sending to the server via the Internet.

Working principle of the robot arm

[0026] Figs.2- 3 illustrate generally a working principle of an example robotic arm according to the present invention. As shown in Fig. 2, the right arm 14R is configured to move the right hand 15 along the keyboards of the piano. In addition to control positions of the hand, the hand’s direction is needed to be controlled constantly in respect of the keyboards of the piano. Therefore, the robot is configured to have three degrees of freedoms (DOFs) in this task, i.e., for controlling the positions along x- direction (horizontal direction), y-direction (vertical direction) of directions of the hand with respect to the piano. With such three degrees of freedoms, typically, a robotic hand may need two actuators to control O’ -position (which is wrist position of the robot arm) in accordance with x- and y- directions, and one additional actuator to control the direction of the hand (not shown in the figures). The inventor finds out that the hand 15R only needs to travel along a straight line along the keyboards of the piano and is far from the piano a d- distance (which is from rotational center O’ to the piano), and the hand is maintained its rotational direction unchanged (i.e., perpendicular to the keyboards of the piano) during working to be able to play the piano. Therefore, for each arm, it is required less than three actuators for controlling the arm to play the piano. Therefore, the present invention provides a robotic arm configured to use only one actuator through a transmission system, wherein the robotic arm is configured to control the robot hand to be movable along a straight line that is parallel with the piano and maintain the direction of hand always perpendicular to t the keyboards of the piano throughout the playing.

[0027] As illustrated in Fig. 3 and Fig. 4A-4B, the example robot arm 14R uses an actuator 22 as above described, to rotate the first link (OA or 25). As used herein, the robot arm comprises links OA and AO’ with rotable centers of shoulder, elbow and wrist of O, A and O’, respectively. Assume that the length of the first link OA is a and is equal to the length of the second link AO’. The first link OA has a big pulley (at position O), and a small pulley (at position A) so that the transmission ratio is 1:2 (i.e., the O-position big pulley has its diameter/number of teeth which is twice that of the A- position small pulley. In other words, when the first link rotates an angle a with respect to the base, the second link will rotate an angle 2a with respect to the first link. The O- position big pulley is mounted to the base 21 and the A-position pulley is mounted the second link (AO’ or 210). When the first link is rotated a a-angle, since the transmission ratio is 1:2, an OAO’-angle between link OA and link AO’ is always 2a. Therefore, when the timing pulley -transmission system has a transmission ratio of 1 :2, only with one actuator 22, the wrist’s motions will follow according to a straight line. Indeed, by geometric demonstration, assume that a case of an angle provided between two links OA and AO’ as O AO' which is always 2a, with a is an angle provided between the first link OA and Oy-axis, we have the equations of calculating of coordinate of point B in the robot coordinate as follows: x(O’) = OA x sin(a)+AO’ x sin(a)= a x sin(a)+a x sin(a)= 2 x a x sin(a) y(O’) = OO’ x cos (a)- OA x cos(a) = 0

(Satisfied the condition that O’ is always located on horizontally straight line).

[0028] Next, the present invention uses one more additional transmission system to maintain the angle between the hand 15R and the second link AO’ is a as mentioned above. The second transmission system includes small pulley (also placed at position A), a big pulley (placed at position O’) so as to ensure the transmission ratio is 2: 1 (i.e., the O’-position big pulley has its diameter/number of teeth which is twice than that of the A-position small pulley, in other words, when an angle between the first and second links is 2a, , an angle between the hand and the second link is always a. The robotic hand 15R is mounted to the big pulley at position O’ . Meanwhile, since the OAO’ -angle is 2a and the transmission ratio is 2:1, the angle between the hand and the second link (AO’) is always a. A relatively relationship among transmission ratios of two above- described transmission systems is to ensure angles a, 2a, a maintained in all links.

[0029] In addition, for actuators as used herein, the actuators’ velocity is not affected depending on the speed of the transmission ratios of two pulley pairs on the first and second links. The actuator can apply directly or indirectly by the first link without changing the structure/configuration of the first and second links. Preferably, the actuator is a direct drive motor.

Robot arm

[0030] Fig. 4A shows a configuration of an example robot arm according to an embodiment of the present invention. The robot arm 14R for playing the piano according to the present invention includes the first link 25, the second link 210, and the hand 15R wherein the first link 25 and the second link 210 have same length, wherein the distal end of the first link 25 is pivotally mounted to the proximal end of the second link 210.

[0031] The robot arm further comprises an actuator 22 mounted to the base 21, a controller 12 is operably mounted to the robot arm, a first transmission system, a second transmission system, a third transmission system. The first transmission system comprises the first pulley 23 mounted at proximal end of the first link 25 at an end of the base 21 through a shaft 215, to drive the said second pulley 26 through a first transmission means 24, the second pulley 26 is mounted at distal end of the first link 25 in the same side with said first pulley 23. Preferably, in accordance with the present invention, a transmission ratio of the first transmission system is 1:2. In accordance with the embodiment of the present invention, the first pulley 23 is mounted to the base 21 by the shaft 215 and a fixture 214, and the fixture 214 is mounted to the base 21 (since it is necessary for the hollowing shaft to go cables through, the pulley 23 should be used to mount to the shaft 215 to fix the pulley 23). The second transmission system comprises a third pulley 27 mounted to proximal end of said second link 25, that is coaxial with the said second pulley 26, and placed at the other side of the proximal end of the second link 210, to drive the fourth pulley 29 through a second transmission means 28, a fourth pulley 29 placed at distal end of the said second link 210 and mounted to the said robot hand 15R. Preferably, in accordance with the present invention, a transmission ratio of the second transmission system is 2:1. In accordance with another embodiment, the present invention further comprises a third mechanical power transmission system comprising a fifth pulley 211 mounted to the actuator 22 to drive a sixth pulley 213 through the third transmission means, the sixth pulley 213 is mounted to the first link 25 at an end of the base 21 and in the same side of the actuator 22, the sixth pulley 213 is coaxial with the first pulley 23.

[0032] Fig. 4B illustrates an arm structure in accordance with another embodiment of the present invention. In accordance with the embodiment the actuator 22 is directly mounted to the first link 25 without using a first transmission system as above described. Therefore, the present invention relates to a robotic arm 14R for playing the piano according to the present invention includes the first link 25, the second link 210, and the hand 15R wherein the first link 25 and the second link 210 have same length, and the distal end of the first link 25 is pivotally mounted to the proximal end of the second link 210. The robotic arm further comprises an actuator 22 mounted to the base 21 and is mounted to the proximal end of the first link 25, a controller is operably mounted to the robot arm, a first transmission system, and a second transmission system. The first transmission system comprises the first pulley 23 pivotally mounted to the actuator 22 through the shaft 215 and mounted at proximal end of the first link 25, to drive the said second pulley 26 through a first transmission means 24, the second pulley 26 is mounted at distal end of the first link 25 in the same side with said first pulley 23. Preferably, in accordance with the present invention, a transmission ratio of the first transmission system is 1:2. In accordance with the embodiment of the present invention, the first pulley 23 is mounted to the base 21 by the shaft 215, which is mounted to a fixture 214, and the fixture 214 is mounted to the base 21. The second transmission system comprises a third pulley 27 mounted to proximal end of said second link 210, that is coaxial with the said second pulley 26, and placed at the other side of the proximal end of the second link 210, to drive the fourth pulley 29 through a second transmission means 28, a fourth pulley 29 placed at distal end of the said second link 210 and mounted to the said robot hand 15R. Preferably, in accordance with the present invention, a transmission ratio of the second transmission system is 2:1.

[0033] In accordance with an embodiment of the present invention, the hand is configured to only travels according to straight line along with keyboards of the piano, wherein the direction of the hand is provided to be perpendicular with the keyboards of the piano.

[0034] In accordance with an embodiment of the present invention, the first, second, third, fourth, fifth, and sixth pulley are timing belt pulleys.

[0035] In accordance with an embodiment of the present invention, preferably, the number of teeth of the first pulley 23 are twice the number of the teeth of the second pulley 26, and the number of the teeth of the fourth pulley 29 are twice the number of the teeth of the third pulley 27. In accordance with another embodiment of the present invention, a diameter ratio of the second pulley 26 to the first pulley 23 is 1 :2, and a diameter ratio of the fourth pulley 29 to the third pulley 27 is 2: 1. In accordance with an embodiment of the present invention, a first, second, third transmission means are belt timings or chains that are fit the first, second, third, fourth, fifth and sixth pulleys, respectively.

[0036] In accordance with an embodiment of the present invention, the robotic arm is provided with coverings to cover outside. Coverings may be made by suitable materials such as plastics, and the like to have protective ability and/or exterior aesthetic looks. Robotic hand

[0037] In accordance with an embodiment of the present invention, Fig. 5 relates to a robotic hand used the robotic arm for playing the piano, wherein the hand comprises a first block 31, a second block 32, a third block 33, and a plurality of fingers 34, 35 in the hand. The first block 31 is provided between the second block 32, and said third block 33, wherein the second block 32 and the third block 33 are configured to be horizontally moved with respect to the first block 31. A plurality of fingers 34, 35 is pivotally provided in respect to the first block 31, the second block 32, and the third block 33, wherein the plurality of fingers 34, 35 has different dimensions on these blocks, wherein a plurality of longer fingers 35 has a length sufficient to press black keys of the piano, a plurality of shorter fingers 34 has a length to press white keys of the piano. The hand further comprises a means for controlling operations of fingers 36, for example, a solenoid, is provided to be mounted on each of the first block 31, the second block 32, and the third block 33, each of fingers is controlled by the means for controlling operations of fingers.

[0038] As shown in Fig. 6-8, the first block 31 of the hand comprises a plurality of bearings holding means 64 that is mounted to the first block 31, each of a plurality of bearings holding means 64 comprises a plurality of linear bearings 63, for example, two linear bearings 63, and is equipped with a position control means 65, for example, solenoids or actuators, of each of the second block 32 and the third block 33. Each of said second block 32, and said third block 33, for example, is provided with a plurality of sliding means 66, which are mounted to each of the second block, and the third block and are slidable with the linear bearings 63. Each of the second block 32, and the third block 33 is mounted to outputs of said position control means 65 of each of said second block 32 and said third block 33 in a way that each of the blocks 32, 33 is horizontally travelable with respect to the first block 31.

In accordance with an embodiment of the present invention, as illustrated in Figs. 9A- 9D, a distance that the second block 32, or the third block 33, can travel with respect to the position of the first block 31 is equivalent to a width of a white piano key. In accordance with an embodiment, the width of a white piano key is around 23.5mm.

[0039] In accordance with an embodiment of the present invention, the first block 31 comprises two shorter fingers and two longer fingers. In accordance with an embodiment of the present invention, each of the second block 32 and the third block 33 comprises one shorter finger and one longer finger.

In accordance with an embodiment of the present invention, a means for controlling operations of fingers 36, for example, solenoid, is placed under the first, second and third blocks 31, 32, 33. As shown in Fig. 10, Fig.10 illustrates the operation of an illustrated finger controlled by a solenoid 51 placed on the second block 32. The solenoid 51 is mounted under the second block 32. The finger 54 is pivotally attached to the second block. When the solenoid 51 controls the shaft 52 of the solenoid incoming and outcoming (attracted by electromagnetic and released by an inside spring), a bolt 53 is coupled with a finger to be able to control the finger to press and release to the piano keys.

Robotic system

[0040] The present invention relates a robotic system for playing the piano wherein the robotic system comprises at least two robotic arms (as shown in Fig. 11), wherein each arm is as above described.

[0041] In accordance with an embodiment of the present invention, the present invention relates a robotic system for playing the piano wherein each arm comprises an above- described robot hand.

[0042] In accordance with an embodiment of the present invention, the robotic system is further configured to mount with a support placed on floor to keep the robot system parallel with the floor. As illustrated in Figs. 12-13, the support frame 41 has a support leg 42 (which may have any different shapes, for example, integral or not integral form, if providing support function) and uses a screw 43 to fix to the piano, creating rigidity for the support frame. The support frame is configured to have a four-bar mechanism, including two same length bars 46 and 47, which are coupled with two revolute joints 44 on the support leg 42, two these revolute joints rotate on the robotic base supporting bar 48. When these revolute joints are rotated, the robotic base supporting bar 48 is moved but always parallel with the support leg 42, meaning parallel with the piano since the support leg 42 is assembled to parallel with the piano. This four-bar mechanism is used to change the robot system from a folded state to a next state, but it is always parallel to the piano. The robotic base supporting bar 48 is configured to connected to the base 21 by two revolute joints 45. After rotating robot around the revolute joints 44 to a desired position parallel with the piano, the robotic system rotates around the revolute joints 45 to raise the robot, transferring the robotic place from vertical to horizontal direction to be ready to play the piano.

[0043] The robotic system in accordance with the present invention can be placed in anywhere along the piano,. When placed in any position fixed with respect to the piano, a user inputs the information to the controller so the robot control exact positions of the robot arms to play music. Alternatively, the robotic system can automatically detect its initial position by using a sound sensor.

Folding of the robotic system when it is not in use

[0044] In accordance with an embodiment of the present invention, one of effects according to the present invention is that the robotic system is compact, user-friendly, easy to use. When it is not in use, the robotic system can be compactly folded so user can quickly switch to play the piano without obstructed by the robot. As illustrated in Fig. 14A, the robotic system is in playing mode with the support frame, when playing is done and the robot needs to be folded, the robot arms may be moved to an initial playing position to be ready for being folded, as illustrated in Fig. 14B. In the first step of folding, the robot hands are folded as shown in Fig. 14C. This step can be done by using an additionally simple lock at wrist joint or elbow joint that is well-known by a skilled person in the art. Then, the robot system is rotated around the joints 45 attached on the frame 41 as shown in Fig. 14D. Finally, folding is completed by rotating the robot system around joints 44 to move to a folded configuration as Fig. 14E. The configuration of the support frame and the steps to fold (an order of steps or removal or addition of a certain step, etc.) can be modified, by a skilled person in the art, if the functions of supporting and folding the robotic system are performed.

Control the robot to play the piano

[0045] As shown in Fig. 15, a robot control system may comprise at least one controller.

[0046] In accordance with the present invention, musical data can be processed by a principal controller (i.e., a processor) which may be, but not limited to, a server, a smartphone, a tablet, or any smart device or any recording medium. The musical data can be processed by some algorithms which process musical data to generate control information to play piano. Those robotic control information may be sent to the robot under a control command file by, but not limited to, wire/wireless communication.

[0047] Alternatively, the robot according to the present invention is further equipped with another controller to receive instructions from the principal controller, without implementation of musical data algorithms. The controller may function as, but not limited to, controlling the robot motions such as positions of the hand and operations of fingers as well.

[0048] In accordance with an embodiment of the present invention, the principal controller and another controller can be integrated into, but not limited to, a single controller. [0049] In Fig. 16, the present invention relates a method for controlling the robotic system to play the piano, in accordance with an embodiment of the present invention, comprising: extracting the music notes that need to be played in a song from a musical file (for example, but not limited to, MIDI or any format), extracting the musical notes in a chord C(i) that must be played at same time, classifying the musical notes of the chord C(i) into musical notes that should be played by each arm, wherein i denotes an ordinal number of an event, saving musical notes which should be played by each arm to a file C.

[0050] Next, as illustrated in Fig. 17, the method further comprises the steps of: searching all combinations of fingers F(j) that can play chord C(i), searching a most suitable combination of fingers F(b) among combinations of fingers F(j) for playing chord C(i), searching the hand positions H(i), corresponding to the most suitable combination of fingers F(b), checking a collision between hand positions H(i) of hand positions; if there is a collision, returns to the step of searching the most suitable combination of fingers F(b) for each position of the hand until there is no collision, saving the hand positions data H(i), the most suitable combinations of fingers data F(b) together with corresponding playing times., wherein i denotes the sequence of the chord, wherein j is the number of positions of fingers that can play chord C(i), [0051] According to another embodiment of present invention, the step of searching a most suitable combination of fingers F(b) is based on the distance that the hands should be moved), preferably, based on the distances that the hands should move in two, three or more later steps. Examples of the invention [0052] For example, a user uses a smart phone, a tablet 11, or any a personal computer) to choose a song to play. For example, the file of the song is saved under MIDI format, or any similar music format containing information of all musical notes that should be played at a specific time of the song. The controller 12 may process the file by abovementioned methods to get all musical notes that should be played. The controller 12 may extract the information of musical notes played at the same time (chord) (C(i)). With a given chord C(i), there may be a plurality of combinations of robot fingers F(j) of the hands 15L, 15R that can play the chord C(i). A simple example is a case that if there is only one musical note that needs to be played, the number of embodiments F(j)hat can play the musical note is same with the number of the fingers of hands 15R, 15L. The controller 12 may search all combinations of the fingers F(j) that can play the chord C(i). Then, the controller finds the best combination of fingers F(b) according to certain criteria. Then, robot hands 15R and 15L will be moved to the position so as the combination of fingers to play the chord C(i). A command is sent to actuate the fingers F(b) that need to be activated to make the sound of the chord C(i). Step by step, the robot plays the song that the user wants to enjoy.

[0053] In another example illustrating the above-described method, as shown in Fig.18, with a song contains four chords named from C(l) to C(4). Let us say R(i) is a hand mode that can play the chord C(i). There could be multiple hand positions that can play the chord C(i), assume that those positions are named as R[i] [j] (equivalent to the above- described F(])). Assume that the hand is at a position R[l] that can play the chord C(l). In next the chord C(2), there may four hand positions that can play the chord C(2). Similarly, there may four hand positions that can play the chord C(3), four hand positions that can play the chord C(4). The problem is to find the best hand positions among R[2].l to R[2].4, to play the chord C(2). To find it, it is necessary to consider routes C(2) to C(3) and C(3) to C(4).

[0054] Therefore, as used herein, assume that the cost to move (cost function) from a hand position i to a hand position (i+1) is CR(i,j), from position the chord C(l) to C(2), there are cost values CR(1,1) to CR(1,4). Similarly, there are sixteen cost values from CR(2, 1 ) to CR(2, 16) from C(2) to C(3). Therefore, the total cost value of a route moving from C(l) to C(4) is (CR(l,k)+CR(2,l)+CR(3,m)), wherein k is a position to play the chord C(2), 1 is a position to play the chord C(3), m is a position to play the chord C(4). [0055] The route with the lowest cost will be the best route, assume that R[2].b is the next best combination with the lowest cost. For example, as shown in Fig. 21, if the route (CR(1,2) + CR(2,5) + CR(3,9) gives the lowest cost, the combination R[2].2 will be chosen as the best option. In accordance with the present invention, the moving distances of the hands from R[i] to R[i+1] are simply used as cost values.

[0056] Although the above-mentioned description are only preferable embodiments, it is not intended to limit the scope of the present invention. Modifications and variations of this invention can be made by those skilled in the art to have similarities or equivalents without departing from the scope or spirit of the invention.