BACKGROUND

[0001] Reading on a computing device may include reading a number of subjects including fiction, non-fiction, and social media. The user’s eyes traverse a display in a direction based on the language being read.

BRIEF DESCRIPTION OF THE DRAWINGS

[0002] FIG. 1 illustrates bookmarking system, according to an example;

[0003] FIG. 2 is a block diagram showing bookmarking system, according to an example;

[0004] FIG. 3 is a flow diagram showing a bookmarking system, according to an example; and

[0005] FIG. 4 is a computing device for supporting instructions for a bookmarking system, according to an example.

DETAILED DESCRIPTION

[0006] Computing devices offer many configurations and form factors for reading materials. A user may utilize a computing device for writing and reviewing documents in their professional or academic roles. A user may also utilize the same device for consuming written content in their personal life such as non-fiction works, fiction works, online content, and social media. For particularly lengthy written works, a user may become tired and distracted. The user may look away from the written work based on a distraction. The user may find it difficult to locate where in the written work they last read. Additionally, a user may read written works that may require multiple sitting to complete, such as novels. A user may close out an application suited for displaying the written work with a bookmark identifying the page on which the user stopped reading. Often the user may return to the written work having forgotten what had occurred in the text prior to that point in the work. With the footprint of displays associated with computing devices increasingly growing; the opportunity for eavesdropping rises.

[0007] Described herein is a bookmarking system. The system utilizes a camera, a display, processor and memory. The memory hosts machine executable instructions that may determine a location associated with the display corresponding to a point at which a user’s gaze is focused. The system may obscure a second location associated with the display, corresponding to a second point at which the user’s gaze is not focused. The system may monitor a change in the point at which the user’s gaze is focused, and responsive to the change surpassing a threshold, display a bookmark at the location.

[0008] FIG. 1 illustrates bookmarking system 100, according to an example. The bookmarking system 100 may include a camera 102, a display 104, and a processor (not shown). A camera 102 may include a red green blue (RGB) camera or an infrared (IR) camera. The camera 102 may be integrated in the display 104 such as a mobile computing device, such as a laptop, tablet, or smart phone. In another implementation the camera 102 may be a standalone device utilizing a communication channel to couple to the processor. One example of a communication channel may be universal serial bus (USB). The camera 102 may be oriented toward a user viewing the display 104 while reading written content rendered on the display.

[0009] The camera 102 may capture a video stream of the user. Within that video stream, the user’s eyes may be captured. The processor may utilize the video stream capture of the user’s eyes to determine a gaze location. Gaze detection may be accomplished by monitoring a difference in the positioning of the pupil of the eye and determining a vector of reflected light based on the movement. Gaze detection may be implemented utilizing a third-party application programming interface (API).

[0010] In FIG.1 the location 108 corresponds to the area of interest of the user’s point of gaze 110. The location 108 may also correspond to the window discussed in relation to a second location: the obscured location 106. The location 108 may be a user configurable area within the document to delineate current area of interest to the user. For example, the location 108 may be predetermined by a user to correspond to two lines before and two lines after the point of gaze 110.

[0011] Within the display 104, the second location, an obscured location 106, may correspond to text that the user has read or has not read. The obscured location 106 may be obscured to prevent a nefarious eavesdropper from physically reading the screen of the user. The obscured location 106 may be obscured by the placement of a filter overlay diffusing the pixels of the letters to prevent the letters from being read. The filter overlay may be applied as the user’s gaze progresses throughout the written content. A window threshold may be determined to minimize distraction for the user, while maintaining security. In one implementation, the window may be four lines of text. The visible lines may correspond to the two lines after the point of gaze and two lines before the point of gaze. The window may be predetermined based on user’s preference. The obscuring effect of the obscured location 106 may be accomplished utilizing application programming interfaces (APIs) which allow the system to access portion of the screen programmatically, to apply various visual effect.

[0012] The point of gaze 110 may correspond to the point within the display 104 bounds that the user is focused. The point of gaze 110 may move systematically based on the language being read. For example, the English language as written, reads from left to right and top to bottom. As such the point of gaze 110 may traverse written content on the display 104 left to right and top to bottom. Other languages may traverse differently. In instances where a user may become distracted, the point of gaze 110 may move outside of the language convention. The point of gaze 110 may move to another location 108 on the display 104, or the point of gaze 110 may leave the display 104 entirely. The latter may occur when the user is distracted from something offscreen. The point of gaze 110 may be determined based on input from the camera 102 as processed by the processor. Point of gaze 110 determination may be available software packages and APIs provided by third parties. The point of gaze 110 may be utilized to track the speed at which a user is reading. By determining the rate at which consistent smooth motion of the point of gaze 110 may be indicative of a reading speed. The system may utilize that rate create statistics regarding the reading, including rate of reading (e.g. words per minute). The rate of reading may be utilized to determine an estimation of time remaining to complete the reading of a section of text.

[0013] The bookmark 112 may correspond to a visual indication. The bookmark 112 may be displayed when a user’s point of gaze 110 deviates or changes from the location 108 for a period of time surpassing a threshold. For example, a point of gaze 110 may move from the display 104 for a very short period of time, such as a second or two. In this example, the threshold may be set to three seconds. The bookmark 112 may not be displayed as the point of gaze 110 returned to the location 108 within the three second threshold. In another example, the point of gaze 110 may move from the display 104 for ten seconds. In this example, bookmark 112 may be displayed as the point of gaze 110 returned to the location 108 after the three second threshold, thereby giving the user a visual indicator as to where they stopped reading prior to looking away from the display. Further, a summary threshold may be utilized for longer term point of gaze 110 movement. For example, the point of gaze 110 may leave the display 104 for a period of long duration, which may be indicative of a user changing tasks or stepping away from the display. In this example, a summary threshold may be set to ten minutes. If the point of gaze 110 does not return to the location 108 prior to the summary threshold, the bookmark 112 may include a summary 114.

[0014]The summary 114 may be a part of the bookmark 112. In at least one implementation the summary 114 may be separate from the visual indication of the bookmark 112. However, the summary 114 may also be integrated into the visual indication of the bookmark 112 as well. The summary 114 may be implemented as a caption as illustrated in FIG. 1. The summary 114 may include a brief summation of the historical text section occurring prior to the location 108 in the display 104. The summary 114 may correspond to main ideas or concepts discussed in text previously read as indicated by the path of the point of gaze 110. The summary 114 may be a pre-determined amount of text (e.g. three paragraphs) prior to where the point of gaze 110 leaves the location 108 and the summary threshold has passed. For example, the pre-determined amount of text may be three paragraphs and the summary threshold is ten minutes. In this example, the point of gaze 110 leaves the location 108 for twelve minutes. A summary 114 corresponding to the previous three paragraphs of text prior to the last reported point of gaze 110 on the display 104 may be generated.

[0015]The summary 114 may be generated utilizing natural language processing (NLP). NLP libraries may be utilized to generate word embeddings to summarize the text according to a pre-defined amount of text requested for the summary 114. In another implementation, the word embeddings may be reorganized into a bullet list identifying main ideas present in the text prior to the last point of gaze 110. The summary 114 may provide the user a brief reminder at a glace that provides context to the text located at the bookmark 112 for when the user resumes reading at that point on the display 104.

[0016] FIG. 2 is a block diagram 200 showing bookmarking system, according to an example. Features illustrated in FIG. 2 may correspond with features illustrated in FIG. 1. As such reference numbers may correspond to both FIG. 1 and FIG. 2 illustrating the relationship between the two examples illustrated in each.

[0017] As described in FIG. 1 , a camera 102 may be utilized for capturing a video stream. The camera 102 may be connected to the processor 202 via a bus such as universal serial bus. The camera 102 may capture a video stream, where the video stream includes a user, particularly a user’s face and eyes. The resolution of the camera 102 may be such as to provide enough pixels to discern high contrast changes in an image to allow for the processor 202 to proficiently perform operations of feature detection. Feature detection may be utilized for determining changes in the movement of a human pupil and thereby extrapolate a point of gaze 110.

[0018] The display 104 may correspond with any display device capable of rendering the bookmark 114 and an obscuring filter. The display 104 may be implemented with varying technologies including but not limited to cathode ray tubes (CRTs), liquid crystal displays (LCDs), organic light emitting diodes (OLEDs), and electronic paper. The display 104 may be implemented as a standalone monitor connected to a system via a display protocol such as DisplayPort™ (DISPLAYPORT is a trademark of Video Electronics Standards Association), or High Definition Multimedia Interface (HDMI™) (HDMI is a trademark of HDMI LICENSING, L.L.C., San Jose, California USA). In another implementation, the display 104 may be an integrated display panel connected to the processor 202 utilizing protocols and signaling across physical interfaces common for that purpose (e.g. Low voltage differential signaling and OpenLDI and FPD-Link).

[0019]The processor 202 of the system 200 may be implemented as dedicated hardware circuitry or a virtualized logical processor. The dedicated hardware circuitry may be implemented as a central processing unit (CPU). A dedicated hardware CPU may be implemented as a single to many-core general purpose processor. A dedicated hardware CPU may also be implemented as a multi-chip solution, where more than one CPU are linked through a bus and schedule processing tasks across the more than one CPU.

[0020]A virtualized logical processor may be implemented across a distributed computing environment. A virtualized logical processor may not have a dedicated piece of hardware supporting it. Instead, the virtualized logical processor may have a pool of resources supporting the task for which it was provisioned. In this implementation, the virtualized logical processor may actually be executed on hardware circuitry; however, the hardware circuitry is not dedicated. The hardware circuitry may be in a shared environment where utilization is time sliced. In some implementations the virtualized logical processor includes a software layer between any executing application and the hardware circuitry to handle any abstraction which also monitors and save the application state. Virtual machines (VMs) may be implementations of virtualized logical processors.

[0021]A memory 204 may be implemented in the system 200. The memory 204 may be dedicated hardware circuitry to host instructions for the processor 202 to execute. In another implementation, the memory 204 may be virtualized logical memory. Analogous to the processor 202, dedicated hardware circuitry may be implemented with dynamic ram (DRAM) or other hardware implementations for storing processor instructions. Additionally, the virtualized logical memory may be implemented in a software abstraction which allows the instructions 206 to be executed on a virtualized logical processor, independent of any dedicated hardware implementation. [0022] The system 200 may also include instructions 206. The instructions 206 may be implemented in a platform specific language that the processor 202 may decode and execute. The instructions 206 may be stored in the memory 204 during execution. The instructions 206 may be encoded to perform operations such as determine a location associated with the display corresponding to a point at which a user’s gaze is focused 208, obscure a second location associated with the display corresponding to a second point at which the user’s gaze is not focused 210, monitor a change in the point at which the user’s gaze is focused 212, and responsive to the change surpassing a threshold, display a bookmark at the location 214.

[0023] In one implementation, a calibration step may occur prior to gaze detection that allows the processor 202 to appropriately track the point of gaze 110 on the display 104. The calibration step may be particularly useful in standalone monitor implementations where the display 104 may be offset from the camera 102 and the user. The processor 202 may receive an input from a user corresponding to a calibration location. The calibration location may be associated with the display corresponding to a calibration point at which the user’s gaze is focused. The calibration point may be generated within a location on the display as a point of reference. The calibration point may be an icon presented within the location to position the user’s head, eyes and gaze so that the camera may capture an image corresponding to that particular calibration position. The processor 202 may receive an image from the camera corresponding to a user’s pupil. As the user may be looking at the calibration point, the point of gaze 110 may correspond to the calibration point, the camera 102 may capture an image of the user’s pupil corresponding to that calibration point on the display. The image capture will allow the processor 202 to compensate for an expected deviation from an anticipated point of gaze 110 within the location. The processor 202 may calculate a plane corresponding to the display based on the input, the calibration location, and the image. The processor 202 may utilize triangulation corresponding on the calibration location, and the image to determine a plane in three-dimensional space where the standalone monitor housing the display exists. The plane may correspond to an orientation of the display 104. The image of the user’s pupil may be utilized to extrapolate a vector corresponding to a positioning of the eye in relation to the point of gaze 110 on the screen. Utilizing the information known about the vector, the point of gaze 110, and the location, the processor 202 may use the calculated plane to determine the correct location of the point of gaze 110 in reference to a display 104.

[0024] FIG. 3 is a flow diagram 300 showing a bookmarking system, according to an example. In describing the features of FIG. 3, reference to features illustrated in FIG.1 may be included. The references allow for better and more concrete understanding of the features illustrated in FIG. 3.

[0025] At 302, the processor 202 determines a location 108 associated with the display corresponding to a point at which a user’s gaze is focused. As described in reference to FIG.1 , a camera 102 captures a video stream and extrapolates a point of gaze 110 corresponding with a location 108 of interest within the display 104.

[0026] At 304, the processor 202 evaluates a historical text section wherein the historical text section corresponds to a text occurring prior to the location on the display. The processor 202 may parse all of the text on the display. The processor 202 may utilize a predetermined summary threshold to determine a lookback buffer corresponding to a historical text section. Utilizing the summary threshold, the processor 202 may traverse the lookback buffer and identify a portion of the historical text section meeting the summary threshold.

[0027] At 306, the processor 202 generates a summary of the historical text section. The processor 202 may apply an NLP algorithm to the portion of the historical text section meeting the summary threshold. The NLP algorithm accessed via third party library may provide a word embedding, or main idea points that occur within the historical text section.

[0028] At 308, the processor 202 displays a bookmark at the location wherein the bookmark comprises a visual indication and the summary. The processor 202 instantiates a graphical visual indication corresponding to a bookmark 112. The visual indication may be an icon or other visual object overlaid on the image rendered on the display 104 to draw the user’s attention, and therefore the user’s gaze. A summary 114 may be presented including the word embedding or main idea points, to provide context to the unread material and to remind the user what the historical text section was about.

[0029] FIG. 4 is a computing device for supporting instructions for a bookmarking system, according to an example. The computing device 400 depicts a processor 202 and a storage medium 404 and, as an example of the computing device 400 performing its operations, the storage medium 404 may include instructions 406-418 that are executable by the processor 202. The processor 202 may be synonymous with the processor 202 referenced in FIG. 2. Additionally, the processor 202 may include but is not limited to central processing units (CPUs). The storage medium 404 can be said to store program instructions that, when executed by processor 202, implement the components of the computing device 400.

[0030] The executable program instructions stored in the storage medium 404 include, as an example, instructions to receive an input from a user corresponding to a calibration location associated with the display corresponding to a calibration point at which the user’s gaze is focused 406, instructions to receive an image from a camera corresponding to a user’s pupil 408, instructions to calculate a plane corresponding to the display based on the input, the calibration location, and the image 410, instructions to determine a location associated with the display corresponding to a point at which a user’s gaze is focused 412, instructions to obscure a second location associated with the display corresponding to a second point at which the user’s gaze is not focused 414, instructions to monitor a change in the point at which a user’s gaze is focused 416, and responsive to the change surpassing a threshold, instructions to display a bookmark at the location 418.

[0031] Storage medium 404 represents generally any number of memory components capable of storing instructions that can be executed by processor 202. Storage medium 404 is non-transitory in the sense that it does not encompass a transitory signal but instead is made up of at least one memory component configured to store the relevant instructions. As a result, the storage medium 404 may be a non-transitory computer-readable storage medium. Storage medium 404 may be implemented in a single device or distributed across devices. Likewise, processor 202 represents any number of processors capable of executing instructions stored by storage medium 404. Processor 202 may be integrated in a single device or distributed across devices. Further, storage medium 404 may be fully or partially integrated in the same device as processor 202, or it may be separate but accessible to that computing device 400 and the processor 202.

[0032] In one example, the program instructions 406-418 may be part of an installation package that, when installed, can be executed by processor 202 to implement the components of the computing device 400. In this case, storage medium 404 may be a portable medium such as a CD, DVD, or flash drive, or a memory maintained by a server from which the installation package can be downloaded and installed. In another example, the program instructions may be part of an application or applications already installed. Here, storage medium 404 can include integrated memory such as a hard drive, solid state drive, or the like.

[0033] It is appreciated that examples described may include various components and features. It is also appreciated that numerous specific details are set forth to provide a thorough understanding of the examples. However, it is appreciated that the examples may be practiced without limitations to these specific details. In other instances, well known methods and structures may not be described in detail to avoid unnecessarily obscuring the description of the examples. Also, the examples may be used in combination with each other.

[0034] Reference in the specification to “an example” or similar language means that a particular feature, structure, or characteristic described in connection with the example is included in at least one example, but not necessarily in other examples. The various instances of the phrase “in one example” or similar phrases in various places in the specification are not necessarily all referring to the same example.

[0035] It is appreciated that the previous description of the disclosed examples is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these examples will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other examples without departing from the scope of the disclosure. Thus, the present disclosure is not intended to be limited to the examples shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.