TECHNICAL FIELD

[0001] The present invention relates generally to the use of robotic devices in education and, in particular, to a robot programmed with applications and software to present course material to a student, monitor the student's behavior and collect data based on the student's attention and performance.

BACKGROUND

[0002] In recent years, technology has improved many aspects of human life. Advances in agriculture, healthcare, energy and communication have eased suffering and allowed humans to live healthier, more productive lives. However, approaches to education have generally not advanced with the pace of technology. Even in the most progressive countries, teachers and educators typically use conventional teaching methods that have remain unchanged for decades.

[0003] Many teachers and other professionals acknowledge the need to improve conventional methods of teaching. They also agree that education should evolve with advances in technology. Only recently have schools begun to utilize online e-learning along with modern interfaces (e.g. phones, tablets). However, much technology has remained underutilized even as computers and internet access have become ubiquitous in industrialized nations. For example, the delivery of lecture material by video is generally outdated.

[0004] Many educational institutions offer lecture material in recorded video formats. Lesson content can be delivered via online platforms with internet access. Some schools stream lectures on-line that students can access with lap-tops or home computers. Colleges now offer degrees that can be earned without entering a classroom. The benefits of video lectures to the student and educator are obvious. The student has the flexibility of watching lectures when convenient without the obligation of commuting to a campus. The school is not obligated to host lectures and can optimize are re-use lecture material that it deems most effective.

[0005] While convenient and cost-effective, lectures presented on video can lead to problems related to student participation and comprehension. Although a student streams lecture material and claims to have participated in a class, there is no effective way for a school to confirm it. A student's physical presence can be acknowledged by requiring periodic input such as answering questions of entering data into a computer. However, educators must rely on conventional methods such as standardized testing to gauge the effectiveness of the lectures and materials. Further, regardless one's participation, students can quickly lose interest and easily get distracted when viewing video lectures.

[00061 Further, studies demonstrate that watching recorded lectures on video is generally an ineffective way of teaching students. Although educators strive to make course content and lectures more interesting to students, their efforts can be futile as a typical teenager may have an attention span of just eight seconds. Further, almost every student above a certain age carries a cellphone which can be a constant source of distraction. A student will review text messages, status updates and participate in social networks at any time given the opportunity.

[0007] Educators have attempted to make video lectures more interactive. For example, U.S. Patent No. 6516340 describes an internet-enabled subscription teaching service system. "Teacher-author stations" allow educators to submit lecture material. Internet-capable lecture client stations have lecture-participation software. However, student interaction and feedback is limited to comments and queries that they submit through the system. Further, there is no way for a teacher or educator to determine whether a student is actively listening to a lecture. A teacher can only monitor whether a student has logged into the system and accessed a file. Moreover, plagiarism can be a problem as there is no way of verifying the sources of a student's work.

[0008] Some have proposed that students merely require supervision to ensure their participation in video lectures under the notion that the teacher is the "boss man" of the classroom. However, an effective teacher must do more than monitor the presence of students. Teachers and educators generally agree that a teacher should strive to coach and motivate students. Further, a teacher should recognize and acknowledge differences among students. Because students have different strengths (different dimensions of intelligence), lessons should be modularized to provide customized training to cater to students who learn in different ways or at different paces. It is difficult, if not impossible, for educators to exercise these skills without direct interaction with students.

[0009] A need, therefore, exists for a system and method to overcome the shortcomings that come with video lectures. Specifically, there is a need for an interface, such as a robot, to engage with a student and improve methods of on-line learning. The system should be capable of recognizing a particular student and monitoring his or her actions and attention while the student observes a lecture. The system should also act as a conduit for the educator to deliver lecture material and collect data about student actions and performance.

SUMMARY OF THE INVENTION

[0010] The invention recognizes that there exists a long felt need for an interface, such as a robot, to implement e-ieaming while monitoring student participation, engaging with students and improving teaching methods. It should also be capable of collecting data on, among other things, student attention, participation and performance. The data can be analyzed to improve conventional methods of teaching and to find educators with particular skills or qualities to contribute to a course or lesson.

[0011] The following summary is provided to facilitate an understanding of some of the innovative features unique to the disclosed embodiment and is not intended to be a full description. A full appreciation of the various aspects of the embodiments disclosed herein can be gained by taking into consideration the entire specification, claims, drawings, and abstract as a whole.

[0012] We describe an interactive learning system comprised of a robot exterior, a central processing unit, a means for identifying the face of a user, one or more scrollable LED touch screen body panels, a means for monitoring a user's level of attention and a means for receiving input from a user and internet and/or blue-tooth connectivity. The interactive learning system can display and store teaching material.

[0013] The interactive learning system can include a three dimensional (3D) hologram face image display. The centra! processing unit can include software for evaluating teacher performance, a keyboard or other means for text input and attendance tracking. The means for receiving input from a user can include voice recognition hardware, face recognition software and sensors to detect human responses. Other aspects can include sensory input for a student (voice and micro facial expression recognition), and text/touch input.

[0014] The means for receiving input from a user can include sensors for detecting the movements of a student. The central processing unit can include one or more storage media for storing course material and data collected from the means for receiving input from a user. The central processing unit can be linked to one or more learning management systems (LMS).

[0015] We also describe a method of operating an interactive learning system using a computer with a robot exterior comprised of the steps of (1 } identifying a student using facial recognition software, (2) displaying stored or streaming lecture to one or more users, (3) identifying the reactions of a user to course material, (4) responding to input from a user, (5) storing the reactions of a user and (6) analyzing data from a user to determine the user's level of participation and level of interest. The method can also include sensory input to determine users' attentiveness and indirectly access his/her motivation and/or the effectiveness of a lecturer.

[0016] The method can include analyzing the reactions of a user to determine the effectiveness of one or more lectures. It can also include analyzing the reactions of a user to determine the effectiveness of course material. The reactions of a user can also be analyzed to determine the user's academic progress. Further, data can be sent to one or more central computers to be compiled with data from other sources to uncover hidden patterns, identify correlations, identify trends and identify user preferences. The method can include the step of monitoring sensory inputs in response to cues from a lecture or lesson. Cues can be, for example, an instruction or question to the student.

[0017] We describe a method of evaluating the effectiveness of a teacher or a method of teaching that includes the steps of (a) providing a computer with at least one input/output module, (b) displaying stored or streaming lecture to a user, (c) conveying cues to the user through said stored or streaming lecture, (d) identifying the reactions of said user to said cues, (e) responding to input from said user, (f) storing the reactions of said user, (g) analyzing data from a user to determine the user's level of participation and level of interest and (h) comparing data from different users to evaluate the level of attention for a given lecture or instructor.

[0018] Further, we describe a method of locating an expert in a vocational or academic discipline comprising the steps of (a) compiling a database of teachers, (b) evaluating and ranking each teacher based on student performance in one or more classes taught by said teacher, (c) identifying a student in a database or learning management system who desires or needs training or instruction on a topic, (d) choosing an effective teacher based on rank and availability and (e) arranging a teaching or tutoring session between the teacher and student.

INTRODUCTION

[0019] The invention includes a robot that can be referred to as "PULA" (Portable Uber Lovable Assistant) which is an "internet-of-things ^M device to assist both educators and students or trainees in both conducting and participating in lessons. The robot can utiiize applications and software to present course material to a student, monitor a student's behavior and collect data based on the student's attention and performance. It can be linked to an education database that contains material prepared by teachers and educators. By observing students, the robot also gathers and record data to be archived and analyzed by educators.

[0020] A first aspect of the invention is a personal robot integrated into an intelligent online based Learning Management System (LMS). St can be integrated into an existing database of student enrolment, academic assessment and quality control systems and processes.

[0021] A second aspect of the invention is deep data analytics. The system can be linked to a distributed ledger database that includes such data as who has attended what courses in which location. This can be used for, among other things, finding topic experts for students.

[0022] A third aspect of the invention is a portable training assistant robot which is integrated alongside an LMS to enhance the educational experience.

[0023] A fourth aspect of the invention is a robot application ("app") that enhances the delivery of streaming or internet based lecture material by observing and interacting with a student. [0024] A fifth aspect of the invention is a control dashboard application {EDIFY SaaS Infrastructure) that is used by the portable training assistant robot.

[0025] A sixth aspect of the invention is a portable training assistant robot that is capable of real time sensory monitoring.

[0026] Other aspects and advantages of the invention will become apparent from the following detailed description, taken in conjunction with the accompanying drawings, illustrating by way of example, the principles of the invention.

BRIEF DESCRIPTION OF THE FIGURES

[0027] The summary above, as well as the following detailed description of illustrative embodiments, is better understood when read in conjunction with the appended drawings. For the purpose of illustrating the present disclosure, exemplary constructions of the disclosure are shown in the drawings. However, the disclosure is not limited to specific methods and instrumentalities disclosed herein. Moreover, those in the art will understand that the drawings are not to scale. Wherever possible, like elements have been indicated by identical numbers.

[0028] FIG. 1 A depicts a perspective view the exterior of the robot.

[0029] FIG. 1B depicts a rear view of the exterior of the robot.

[0030] FIG. 1C depicts a side view of the exterior of the robot.

[0031] FIG. 2 illustrates an example of an interaction between a teacher and student, both of who are using the invention according to at least one embodiment.

[0032] FIG. 3 is a graphical depiction of the series of steps used to organize a lesson or teaching session with a student and an expert in a particular subject. [0033] FIG. 4 is a graphical depiction of a method for monitoring a student's participation and level of attention according to one embodiment.

DETAILED DESCRIPTION OF THE INVENTION

Definitions

[0034] While the invention is primarily described for use for education, it is understood that the invention is not so limited and can be used in business, advertising, marketing and other various industries. Other applications include, for example, but not limited to, using the invention in professional/career training, foreign language studies, instruction for operating vehicles/machinery and for entertainment purposes.

[0035] Reference in this specification to "one embodiment/aspect" or "an embodiment/aspect" means that a particular feature, structure, or characteristic described in connection with the embodiment/aspect is included in at least one embodiment/aspect of the disclosure. The use of the phrase "in one embodiment/aspect" or "in another embodiment/aspect" in various places in the specification are not necessarily all referring to the same embodiment/aspect, nor are separate or alternative embodiments/aspects mutually exclusive of other embodiments/aspects. Moreover, various features are described which may be exhibited by some embodiments/aspects and not by others. Similarly, various requirements are described which may be requirements for some embodiments/aspects but not other embodiments/aspects. Embodiment and aspect can in certain instances be used interchangeably.

[0036] The terms used in this specification generally have their ordinary meanings in the art, within the context of the disclosure, and in the specific context where each term is used. Certain terms that are used to describe the disclosure are discussed below, or elsewhere in the specification, to provide additional guidance to the practitioner regarding the description of the disclosure. For convenience, certain terms may be highlighted, for example using italics and/or quotation marks. The use of highlighting has no influence on the scope and meaning of a term; the scope and meaning of a term is the same, in the same context, whether or not it is highlighted. It will be appreciated that the same thing can be said in more than one way.

[0037] Consequently, alternative language and synonyms may be used for any one or more of the terms discussed herein. Nor is any special significance to be placed upon whether or not a term is elaborated or discussed herein. Synonyms for certain terms are provided. A recital of one or more synonyms does not exclude the use of other synonyms. The use of examples anywhere in this specification including examples of any terms discussed herein is illustrative only, and is not intended to further limit the scope and meaning of the disclosure or of any exemplified term. Likewise, the disclosure is not limited to various embodiments given in this specification.

[0038J Without intent to further limit the scope of the disclosure, examples of instruments, apparatus, methods and their related results according to the embodiments of the present disclosure are given below. Note that titles or subtitles may be used in the examples for convenience of a reader, which in no way should limit the scope of the disclosure. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains. In the case of conflict, the present document, including definitions, will control.

[0039] The term "App" or "application" refers to a self-contained program or piece of software designed to fulfil a particular purpose, especially as downloaded by a user to a mobile device.

[0040] The term "bluetooth" refers to a standard for the short-range wireless interconnection of mobile phones, computers, and other electronic devices. [0041] The term "control dashboard application" refers to a software-based control panel for one or more applications, network devices or industrial machines. Dashboards may display simulated gauges and dials that look like an automobile dashboard or a factory assembly line, or they may show business graphics such as pie charts, bar charts and graphs.

[0042] The term "deep data analytics" or "big data analytics" refers to the process of examining large data sets containing a variety of data types (i.e., big data) to uncover hidden patterns, unknown correlations, market trends, customer preferences and other useful business information.

[0043] The term "input/output module" or "I/O module" refers to a component that is connected to the computer system on one end and one or more input/output devices on the other. A device which is connected to an I/O module of computer is called a "peripheral device."

[0044] The term "Learning Management System" or "LMS" refers to a software application for the administration, documentation, tracking, reporting and delivery of electronic educational technology (also called e-learning) courses or training programs.

[0045] Other technical terms used herein have their ordinary meaning in the art that they are used, as exemplified by a variety of technical dictionaries. The particular values and configurations discussed in these non-limiting examples can be varied and are cited merely to illustrate at least one embodiment and are not intended to limit the scope thereof.

Description of Preferred Embodiments

[0046] The invention includes three main components: (1) a robot exterior, (2) software or a series of software applications (Apps) and (3) a control dashboard application (EDiFY SaaS Infrastructure). An embodiment of the invention is, together with an LMS, to assist in providing analytical data to an instructor/teacher, so that the instructor/teacher can concentrate on coaching, motivating and inspiring, rather than spending time and effort on content delivery and monitoring.

[0047] The invention includes a robot with input/output capabilities along with interconnectivity to a database and a learning management system. However, it is more than a content delivery system. The focus of the invention is to coach and inspire, rather than just to deliver content. It can assume the roles of a friend, a coach and an empathizer to encourage students to be intellectually curious. The benefits of an integrated LMS include the achievement of productivity and efficiency, wherein content and curriculum can be organised, stored, managed and most importantly shared through electronic online systems.

[0048] The invention can be offered as part of a Learning Management System (LMS) system to get students away from cell phones and personal computers, and improve student participation while, at the same time, serve as a device for data analytics. The invention can also function as a training aide for educators to allow them to interact (through one-to-many mode), via internet, with audiences in any part of the world.

[0049] The invention can include interactive features (e.g. the ability to customise the avatar head, keying in texts to pose as Q&A questions and rating an educator to give feedback) which can help to engage students better so that they can concentrate on listening and absorbing the content delivered by lecturers. It can also include backend support to analyze students' attention span along with tools to attract students to join study groups or participate in certain courses.

[0050] One or more sensory input modules can be used to to determine a user's attentiveness - and indirectly access his/her motivation or the effectiveness of the lecturer. This can assist a teacher or education professional in keeping students motivated as well as employing/emphasizing proven methods of keeping students engaged.

[0051] Another aspect of the invention is real time sensory monitoring. A teacher or administrator can monitor the use of the robot from a remote location. He or she can confirm that a student is participating, thus preventing the possibility of cheating or plagiarism. The teacher or administrator will also have access to data compiled from each course and session of which a student has participated. Further, the data can be used to find the topic experts residing anywhere in the world.

[0052] The invention can have a broad impact on schools, colleges and education systems. For example, institutions continue to face pressure to improve their services while reducing costs. The invention can help address these market challenges as it allows schools and colleges to include online e-learning along with an interactive robot interface into their curriculums. The invention offers an alternative to classroom lectures while eliminating the shortcomings of conventional methods of e-learning. Further, the data compiled by the invention can be used to address problems with conventional methods as well as to acknowledge successes.

Exterior and Hardware

[0053] FIG. 1A depicts a perspective view of the exterior of the invention in a preferred design. Some components of the invention include:

• a head/body with camera with face recognition technology,

• one or more scrollable LED touch screen body panels to display or input information,

• one or more microphones for voice input,

• a three dimensional (3D) hologram face image display,

• servo wheels for movement, • one or more Internet modules for external communication,

• one or more Bluetooth modules for external communication,

• internal software for tasks that include (i) teacher evaluation (ii) text input (to pose questions) to trainers (iii) attendance tracking,

• accompanying application ("app") for events/training booking and scheduling images and sounds, via desktop and mobile channels.

[0054] FIG. 1 B depicts a rear view and FIG. 1 C depicts a side view of the exterior of the invention. While functionality of the wheels and arms is preferred, it is not essential for use of the invention.

[0055] FIG. 2 depicts the use of the invention for streaming a lecture to a student. The video and audio of the teacher 110 is relayed in real time to the student 111. The teacher can receive input from the student. For example, a student can use a touch screen or external keyboard or other device to type questions. The questions can be visualized by the teacher on a split screen of the robot. In another embodiment, a camera on the student's robot relays real time video of him to the teacher. This allows the teacher to visualize the student's level of attention and respond to a student's gestures or appearances. The invention can also be used for streaming to multiple students, each through a robot at his or her location.

[0056] FIG. 3 is a graphical depiction of the series of steps used to organize a lesson or teaching session with a student and an expert in a particular subject. The system can be referred to as "GOKU" and can be used when a student seeks advice or training on a particular topic. It can be defined as a recommendation engine that is part of a database of experts and teachers.

[0057] The system can be used to locate an expert in a vocational or academic discipline for a given lesson. For example, the LMS can include a database of teachers who are skilled at teaching trigonometry. Each teacher can be evaluated and ranked based on student performance in one or more classes (i.e. trigonometry) taught by a teacher. A teacher can aiso be evaluated based on ievels of student attention and compliance.

[0058] The system can also be used to identify a student who needs training on instruction on a particular topic. The student can be one who is underperforming in a course (i.e. trigonometry) or one with a desire or interests on a subject. The system can choose one or more teachers in the database deemed to be effective for the student based on rank and availability. Further, the system can arrange a teaching or tutoring session between the teacher and student.

[0059] FiG. 4 is a depiction of the use of the invention to gauge the level of attention of a student. The robot can display or broadcast a stored or streaming lecture to a user. The lecture can conveying "cues" to the user to instigate a response. For example, the student can be told to input an answer, take notes on a matter or draw a shape. Sensors on the robot can detect activity (or iack of activity) to evaluate the student's level of attention and compliance. If a student's activity falls below a certain threshold, he or she will be deemed to be non-attentive.

[0060] This data can be stored and relayed to an instructor or educator. In one embodiment, the robot is programmed with a series of actions intended to improve a student's level of attention. For example, the robot can present questions to the student or inquire on the student's mood or desire to take a respite from the session.

[0061] Data from multiple students and lectures can be compiled and compared among teachers. A teacher with a higher level of student attention and compliance can be ranked higher than a teacher with a lower level of student attention and compliance. Thus, a teacher or lecture can be deemed ineffective if a high number of students display low Ievels of attentions and compliance. The criteria used to determine a teacher's rank and success can also include conventional criteria such as grades and scores on standardized tests. [0062] Another aspect of the invention is a deep data analytics. The invention can gather data for analysis to address two important questions in education, "What motivates students to come to school and study?" and "How can a curriculum be adapted and changed to meet the needs of our students?" Among other observations, the invention can observe and record:

• the amount of time spent by a student to complete a course,

• the grade/score of the student,

• the proportion of time the student was engaged {i.e. alert) during a lesson (to understand student engagement and effectiveness of educators),

• Text analysis to discover key words (provided by students in open ended questions or answers) which will allow teachers to further improve curriculum.

Use of software and database of the invention

[0063] The system can be implemented using any of a number of different platforms, such as the web, email, smartphone, and the like, or any combination thereof. In one embodiment, the system utilizes facial recognition components to identify a student, then monitors the student's actions and attention while presenting lecture material through a screen and holographic avatar head. In a preferred embodiment, data storage is linked to one or more cloud based data storage systems. Local storage is use primarily for environmental (e.g. wifi password) and robot setting purposes.

[0064] A built in control dashboard application can set environmental parameters for the robot to operate, as well as user preferences. This can be supported also by an external application ("PULA APP") for purposes of communication with users and setting/scheduling/editing of "hang out" sessions involving trainers and trainees. The robot can also perform the role of the instructor. For example, it can alert the student when he is not concentrating (e.g. talking to others, playing with his phone, sleeping). Working Example - Use of PULA to Stream a Lecture

[0065] Schools face enormous challenges in attracting students to enroll and thereafter motivate them to participate in classes while maintaining good study habits. Educators must continue to develop innovative educational curriculum and creative teaching resources. As technology advances, traditional methods of teaching will become less relevant.

[0066] The conventional approach to classroom teaching essentially involves a teacher or lecturer who recites didactic information to a group of students. This may be followed by some questions and clarification and ideally, some discussions and role play among the students. A teacher can use a number of teaching resources hardcopies, digital assets), and multimedia devices {e.g. projectors and speakers) in this endeavor.

[0067] This mode of teaching is gradually becoming less relevant as hand-phones are super computers can deliver a seemingly unlimited amount of content (including virtual reality). The educator should be concentrating more on facilitation, and more important, motivating her students to reach their maximum potential. The trainer should come to class to coach and inspire, rather than to "deliver content." Hence, efforts should be focused on taking "content delivery" away from the lecturer, so that the lecturer can concentrate on the most important ("human") aspect of the educational experience — providing guidance, counselling, motivating and inspiring students. Further, the robot can act as the interface and suggest advancement to another set of materials after progression analysis.

Activation of the Robot

[0068] A student can begin a lesson by activating one or more switches on the robot. In the alternative, the robot can include a sensor that is voice or sound activated. This can turn on the three-dimensional hologram face display and the robot can display or convey a greeting to the student. The robot can utilize facial recognition to confirm the identity of a student and begin measuring the student's level of attention and engagement. Thereafter, it can begin a session or continue a lesson, lecture or coursework from a previous session with the student.

[0069] When the identity of a student is confirmed, the robot can access electronic records from the student. The robot can review tasks or a digital calendar for the student and convey instructions or reminders. The student's records can be used to determine coursework or lectures to be conveyed to the student.

Displaying/Streaming a Lecture

[0070] Depending on a curriculum, a student can tune into a streaming lecture on a particular date and time. In the alternative, the student can choose a lecture from a database or the robot can inform the student that a lecture is assigned from a curriculum or course instructor. Thereafter, the student can observe the robot as the lecture is conveyed on a screen. The robot can also perform to assume the role of the instructor through the combined use of its sensory input and sensory outputs.

[0071] For example, the robot maintains perceptive capabilities. It continues to monitor the actions of the student to ensure that he or she continues to stay actively involved with the lecture. The robot can begin one of several interaction sequences if the student begins to display signs of boredom. Another interaction sequence can be activated if the student is distracted by an external device {e.g. cell phone or computer).

Gauging Student Achievement

Ϊ0072] The robot can periodically use one or more cues to initiate a response from the student (e.g. sensory input). The responses can be used to determine a student's level of participation and understanding. For example, motion sensors can detect if a student is still/stationary and likely to be sleeping. The sensors can also detect a activity at a time when the student should be active (e.g. writing, typing or actively participating). If the student remains passive, the robot can send a cue or signal to the student to initiate a response. The robot can thereafter continue to monitor the student and periodically send cues or signals.

[0073] The robot can also convey conventional test questions either on a screen or through a voice module. The response can be used to determine a student's level of comprehension and whether sections of a lecture should be repeated. Additional materials on a subject can be assigned to the robot through the L S or course instructor. If the student has demonstrated a threshold level of proficiency, then the robot can advance to another set or materials.

[0074] A school and/or teacher can also utilize the invention to improve administrative and recording capabilities. Improvements over conventional methods include:

• improved curriculum development using data analytics and "free text" inputs when students pose questions or answer questions,

• real-time student feedback from the robot input/output modules,

• real-time student feedback and evaluation that can be automatically consolidated and analyzed,

• evaluation of a lecturer based on an analysis of students' attention span,

• student attendance in real time than can include the time of a student's physical presence as well as her "active" participation.

The data can be collected and used for teacher, curriculum and course appraisal. Further, the data can be used more broadly to evaluated schools and approaches to education in districts and regions.

[0075] It will be appreciated that variations of the above disclosed and other features and functions, or alternatives thereof, may be combined into other systems or applications. Also, various unforeseen or unanticipated alternatives, modifications, variations or improvements therein may be subsequently made by those skilled in the art which are also intended to be encompassed by the following claims.

[0076J Although embodiments of the current disclosure have been described comprehensively, in considerable detail to cover the possible aspects, those skilled in the art would recognize that other versions of the disclosure are also possible.