Background

[0001 ] Applications running on computing devices can be used by users to perform a variety of activities. Examples of activities including using a travel booking application to make travel reservations, using an online review application to find restaurants, using an application to perform language translations, and so forth.

Brief Description Of The Drawings

[0002] Some implementations are described with respect to the following figures.

[0003] Fig. 1 is a block diagram of an example arrangement that includes computing devices and an analytics server according to some implementations.

[0004] Fig. 2 is a flow diagram of an example analytics process according to some implementations.

[0005] Fig. 3 is a schematic diagram showing an example sequence of screens represented by event data that can be processed by an example analytics engine according to some implementations.

[0006] Fig. 4 is a flow diagram of an example process of identifying a navigation hub screen, according to some implementations.

[0007] Fig. 5 is a graph showing example metric values computed for various screens, according to some implementations.

[0008] Fig. 6 is a flow diagram of an example process of identifying a task beginner screen, according to some implementations.

[0009] Fig. 7 is a schematic diagram of an example segment that includes multiple task flows, according to some implementations. [0010] Fig. 8 is a block diagram of an example system according to some implementations.

[001 1 ] Fig. 9 is a block diagram of an example storage medium according to some implementations.

Detailed Description

[0012] Different users can run instances of an application on different types of computing devices (e.g. smart phones, tablet computers, notebook computers, desktop computers, etc.), in different operating system environments, using diverse network conditions, and across different geographic locations. An application analytic service can be employed by a developer of an application to monitor user interactions with instances of the application run on many computing devices. In some cases, the application analytic service can provide relatively rudimentary metrics based on the monitored user interactions, where examples of such

rudimentary metrics include the following: a fraction of users who waited more than seconds after clicking a button of a screen presented by the application, a number of crashes reported in response to clicking the button, and so forth. Such metrics may not be sufficient for the developer to effectively update the application to improve user experience in running the application.

[0013] In accordance with some implementations of the present disclosure, analytics techniques or mechanisms are provided to derive deeper insights based on event data collected from monitoring interactions between users and instances of an application. The event data is of events representing user actions made with respect to multiple screens presented by the application. A "screen" can refer to a user interface (Ul) object that can be displayed in a display device and that includes information content as well as a control element (or multiple control elements) that is (are) activatable by a user to interact with the application. Examples of user actions made with respect to screens can include entering text into a text field, selecting a control item (such as a menu item), selecting a control button, or any other type of user action that interacts with a screen. [0014] Based on transitions between screens that are responsive to user actions represented by the event data, a task flow that includes multiple user actions can be extracted. In some implementations, multiple task flows can be extracted based on the event data. A task flow can refer to a sequence (temporal sequence or spatial sequence) of user actions taken to perform a task.

[0015] Although reference is made to collecting event data representing user actions made with respect to screens presented by an application, it is noted that in other examples, event data representing user actions made with respect to screens presented by multiple applications can be collected, and analytics techniques or mechanisms according to some implementations can be used to determine task flows including multiple user actions made with respect to the multiple applications.

[0016] Examples of deeper insights that can be provided by analytics techniques or mechanisms according to some implementations of the present disclosure include an amount of time (e.g. average time) spent by users in completing a reservation or some other activity, a number of click actions (e.g. average number of click actions) taken by users to complete a reservation or some other activity, a screen of the application that is most time consuming, a screen of the application that is most likely to be abandoned by users, a frequency of user interface errors, and so forth.

Although some examples of deeper insights are provided above, it is noted that in other examples, other types of insights can be derived using analytics techniques or mechanisms according to some implementations.

[0017] Fig. 1 is a block diagram of an example arrangement that includes an analytics server 102 and various computing devices 104 that each executes an instance of an application 106. Examples of the computing devices 104 include any or some combination of the following: a desktop computer, a notebook computer, a tablet computer, a smart phone, a game appliance, a television set-top box, a wearable device (e.g. a smart watch, smart eyeglasses, etc.), a vehicle, and so forth. The analytics server 102 can be implemented as a computer, or as a distributed arrangement of multiple computers. [0018] Although just one application 106 is shown running in each computing device 104, it is noted that in other examples, a computing device 104 can execute multiple different types of applications. Examples of the different types of

applications include any or some combination of the following: an application to make online reservations, an application to find restaurants, a word processing application, an email application, a text messaging application, a game application, and so forth. Generally, an application can refer to machine-readable instructions that can be invoked by a user or other entity (e.g. a machine or machine-readable instructions) and executed by a hardware processing circuit of the computing device 104 to perform target tasks. Examples of a hardware processing circuit can include any or some combination of the following: a microprocessor, a core of a multi-core microprocessor, a microcontroller, a programmable gate array (PGA), an application- specific integrated circuit (ASIC) device, and so forth.

[0019] The analytics server 102 includes an analytics engine 108 that can perform analytics according to some implementations of the present disclosure. As used in this disclosure, an "engine" can refer to a hardware processing circuit or a combination of a hardware processing circuit and machine-readable instructions executable on the hardware processing circuit.

[0020] In examples according to Fig. 1 , each computing device 104 includes a monitoring agent 1 10, which can be part of the respective application 106 or can be separate from the application 106. The monitoring agent 1 10 can include machine- readable instructions that are executable on a hardware processing circuit in the computing device 104, to collect event data associated with user interactions with screens 1 12 presented by the application 106 in a display device 1 14 of the corresponding computing device 104.

[0021 ] The monitoring agent 1 10 monitors for user interaction with a screen 1 12 presented by the application 106. For each action made by the user that includes an activation of a control element of a screen 1 12 or entry of information into a screen 1 12 or other user interaction with a screen 1 12, the monitoring agent 1 10 can collect an event record, which can include the following information, as examples: the time at which the action was taken, an identifier of the action, and an identifier of a user that took the action. In other examples, other types of information can be included in an event record.

[0022] Table 1 below provides an example of an event data stream of event records that can be provided by the monitoring agent 1 10. The multiple event records of the event data stream of Table 1 are represented by corresponding rows of Table 1 , where each event record is represented by a corresponding row in Table 1 .

Table 1

[0023] Each row in Table 1 includes three columns, where the first column corresponds to the time at which an action was taken, the second column includes an action identifier (ID), and the third column includes a user ID.

[0024] Each action ID provides an indication of a screen 1 12 against which the corresponding action was made, as well as the corresponding action itself. For example, a first action identifier can specify that input text was entered into a text box of a first screen, a second action identifier can indicate that a selection was made in a control field of the first screen, a third identifier can indicate that a control button was activated in the first screen, a fourth action identifier can indicate that a different control element was activated in a second screen, and so forth. [0025] Table 2 is an example of an actions table (or other data structure) that includes information pertaining to what each action ID means.

Table 2

[0026] In the actions table (Table 2), the first column includes the action ID, the second column indicates the screen against which the action indicated by action information in the third column was made. For example, action ID a ₂ indicates that an input (an action) of the origin of a flight was made in a Flight Search screen.

[0027] Although specific formats of event records and an actions table (or other data structure) are shown in Tables 1 and 2, it is noted that in other examples, the event records and actions table (or other data structure) can have other formats and can include other information.

[0028] The event records are sent by the monitoring agents 1 10 executing in the computing devices 104 over a network 1 16 to the analytics server 102, for storage in an event log 120 that is stored in a storage 122 of the analytics server 102. Although the storage 122 is shown as being included in the analytics server 102, it is noted that in other examples, the storage 122 can be separate from the analytics server 102. The storage 122 can include a storage device, or multiple storage devices.

[0029] In accordance with some implementations of the present disclosure, the analytics engine 108 processes the event data contained in the event log 120, where the event data includes the event records corresponding to events collected by the monitoring agents 1 10. The analytics engine 108 extracts, based on the event data, task flows 124 from the event data, where each task flow includes multiple user actions taken with respect to screens 1 12 presented by the application 106. The event data can be unlabeled event data (unlabeled with respect to tasks that the events of the event data correspond to). In addition, the extraction of the task flows by the analytics engine 108 can be performed without having to modify the application 106, such as instrumenting the application 106 with special code to identify tasks.

[0030] As further shown in Fig. 1 , the extracted task flows 124, or information derived from the extracted task flows 124, can be provided to an application developer station 126, which is a computer (or computers) associated with a developer of the application 106. Examples of information derived from the extracted task flows can include statistics or other metrics that can be used by an application developer to develop a future version of the application 106. As explained further below, the information derived from the extracted task flows 124 can be according to a specified data model. Also, in some examples, a query interface (not shown) can be provided to allow an application developer at the application developer station 126 to submit queries to the analytics server 102 to seek specific information derived from the extracted task flows 124.

[0031 ] Table 3 lists various example statistics that can be computed for a given task. Although specific examples are provided, it is noted that in other examples, other types of statistics can be derived.

Table 3

Statistics

1 Time spent in completing the task

2 Number of actions executed by users to complete the task

3 Dwell time spent for each screen in the task path

4 Actions executed on each screen in the task path 5 Hours of day when the task is executed.

[0032] Fig. 2 is a flow diagram of an example analytics process that can be performed by the analytics engine 108 according to some implementations. The analytics engine 108 receives (at 202) event data of events representing user actions made with respect to screens presented by an application (e.g. the

application 106). The received event data can be received from the event log 120, or alternatively, can be received directly from monitoring agents 1 10 in the

computing devices 104 without first being stored in the event log 120.

[0033] The analytics engine 108 identifies (at 204), based on the event data, a starter screen from among the screens presented by the application 106. The starter screen contains a user-activatable control element to begin use of the application. The starter screen can include a navigation hub screen or a task beginner screen. An explanation of a navigation hub screen and a task beginner screen is provided further below.

[0034] The analytics engine 108 extracts (at 206), based on transitions between screens including the identified starter screen and other screens of the screens presented by the application, a task flow that includes multiple user actions. The transitions between screens are responsive to the multiple user actions.

[0035] Fig. 3 shows an example of sequence of screens visited by a user, which can be derived from the received event data collected by the monitoring agent(s) 1 10. Each screen in Fig. 3 is represented by a corresponding box, e.g. Main Activity screen, Flight Search screen, and so forth. Although unclear from a quick glance, closer inspection reveals that the user is trying to accomplish the following four tasks in this example: (1 ) search for a flight, (2) read passenger details for a booked flight, (3) read baggage information for a booked flight, and (4) book a hotel. [0036] In accordance with some implementations of the present disclosure, the analytics engine 108 is able to identify task flows corresponding to the foregoing four tasks based on the received event data. The analytics engine 108 can identify a start and end point of a task, overlapping sub-paths, and/or any shortcuts present in a user interface (Ul) presented by the application 106.

[0037] An execution path of a task (also referred to as a "task execution path") includes an ordered sequence of screens that are visited to complete a task. A sub-path refers to a subset of a task execution path.

[0038] In the example according to Fig. 3, in response to a user opening the application 106, the application 106 presents a Main Activity screen 302. The user then searches for a flight (using a Flight Search screen 304), and views the results (in a Flight List screen 306). In this example, the Flight Search screen 304 marks the start of task (1 ) noted above (task to search for a flight), and the Flight List screen 306 marks the end of task (1 ). In this example, the task execution path for task (1 ) is Main Activity→ Flight Search→ Flight List.

[0039] Next, the user uses a Menu screen 308 to transition to task (2) above (task to read passenger details for a booked flight). Task (2) starts with a Booked Flights screen 310, transitions to a Flight Details screen 312, and ends with a Passenger Details screen 314. Next, the user transitions to task (3) above (read baggage information for a booked flight), by backing up from the Passenger Details screen 314 to a Flight Details screen 316 and then proceeding to a Baggage

Information screen 318. Note that task (3) is only partially captured in the sequence depicted in Fig. 3. Task (3) ends at the Baggage Information screen 318, but starts at a Booked Flights screen, which then transitions to the Flight Details screen 316 before ending at the Baggage Information screen 318. Tasks (2) and (3) have overlapping sub-paths in that each starts at a Booked Flights screen and transitions through a Flight Details screen before ending at the Passenger Details screen 314 and the Baggage Information screen 318, respectively. [0040] Finally, the user follows a shortcut in the Ul presented by the application 106 to a Hotel Search screen 320 to begin task (4) (book a hotel), which transitions through a Hotel Details screen 322, a Book Hotel screen 324, a Checkout screen 326, and ends at a Confirmation screen 328. A shortcut can be in the form of a link that is selectable by a user, or some other control element in the Ul.

[0041 ] Using information derived from the captured task flows 124 as captured by the analytics engine 108 according to some implementations of the present disclosure, an application developer can perform various analyses. For example, the application developer can perform an evaluation of the Ul of the application 106. The application developer can evaluate the efficiency of the Ul by visualizing distributions of a number of actions or screen transitions taken to complete a task. Additionally, the application developer can analyze the time spent to complete a task or the fraction of time spent on individual screens. These metrics can help the application developer in identifying bottleneck screens or actions impacting the overall performance of execution of a given task.

[0042] In further examples, the application developer can also perform contextual crash analysis. In addition to stack traces of crashes encountered during execution of the application 106, additional information can be added to crash information, including user actions as well as task execution paths taken. The added information pertaining to user actions and task execution paths enriches the crash information by providing contextual information.

[0043] In additional examples, the application developer can perform task abandonment analysis. Task flows can be analyzed to compute the abandonment rate at various screens. The application developer can identify screens where users deviate from the expected task execution path.

[0044] In other examples, the application developer can perform task popularity analysis. The application developer can identify the more popular tasks and can update the application to prioritize the efficiency in execution of the more popular tasks. Conversely, less popular tasks can be analyzed to establish causality between unpopularity and Ul inefficiencies.

[0045] The application developer can also perform user categorization. For a multi-task application used by different users for different purposes, users can be categorized based on the distribution of tasks accessed over a period of time. The identified user categories can be utilized to optimize respective Uls or to introduce a new set of features targeted to those categories.

[0046] As discussed above, task 204 in Fig. 2 identifies a starter screen, which can be a navigation hub screen or a task beginner screen.

[0047] A navigation hub screen provides an entry point of the application and allows a user to choose from among multiple task options in the navigation hub screen. A "task option" can refer to an option selectable by a user to perform a respective task; different task options correspond to different tasks. A navigation hub screen can be a first actionable screen presented by the application upon launch of the application. An "actionable" screen refers to a screen in which a user can make a selection the application to respond. An example of such a first actionable screen is the Main Activity screen 302 shown in Fig. 3.

[0048] Another example of a navigation hub screen in Fig. 3 is the Menu screen 308. The Menu screen 308 can be selected by a user from any screen presented by the application to allow the user to gain choose from multiple task options.

[0049] While a navigation hub screen provides a starting point for a user, a task beginner screen is a first screen that begins a task. In other words, the task beginner screen is the first screen encountered after beginning a task. For example, the Flight Search screen 304 is the task beginner screen for task (1 ) of Fig. 3 (search for a flight), the Booked Flights screen 310 is the task beginner screen for each of tasks (2) and (3) (read passenger details for a booked flight, and read baggage information for a booked flight, respectively), and the Hotel Search screen 320 is a task beginner screen for task (4) (book a hotel). In some examples, a task beginner screen is immediately accessible from a navigation hub screen.

[0050] In some cases, a user may switch between different tasks by using a navigation hub screen as an intermediate screen (e.g. the Menu screen 308 in Fig. 3). The new task execution begins by transitioning to a task beginner screen. In other cases, a user may switch between tasks without going to a navigation hub screen, such as switching between task (2) and task (3) in Fig. 3, or switching between task (3) and task (4) in Fig. 3.

[0051 ] Navigation Hub Screen Identification

[0052] Fig. 4 is a flow diagram of an example process of identifying a navigation hub screen, in accordance with some implementations. The example process of Fig. 4 can be performed by the analytics engine 108, for example. The process of Fig. 4 includes learning (at 402) a screen connectivity graph 403, which is a directed graph that models screen transitions that may occur during use of an application. The screen connectivity graph 403 includes a set of nodes corresponding to each screen in the application, and edges each representing a transition from one screen to another screen.

[0053] The sequence of screens presented by an application that are visited in response to user actions can be obtained based on event data stream (e.g. such as represented by Table 1 above) and the actions table or other data structure (e.g. Table 2 above). The combination of the information in the event data stream and the actions table or other data structure allows the analytics engine 108 to detect distinct screens of the application that have been visited, as well as transitions between screens. A node is added to the screen connectivity graph 403 for each screen observed in the event data, and a directed edge is added for each detected transition between screens.

[0054] An intuition behind navigation hub identification is that there exists a directed path in the screen connectivity graph 403 from a navigation hubs screen to any screen in a task execution path. A directed path can refer to a sequence of one or multiple edges between or among nodes of the screen connectivity graph 403. Also, a user can navigate back to a navigation hub screen from any screen in the task execution path by retracing their steps using a back button. In other words, navigation hubs act as central nodes in the screen connectivity graph 403 connecting the screens along one task execution path to another task execution path. A centrality of a node is based on a relationship of the node with respect to other nodes.

[0055] The process of Fig. 4 includes computing (at 404) a metric indicating the centrality of each node of the nodes in the screen connectivity graph 403. The centrality of a node in the screen connectivity graph 403 can be expressed using any of various different metrics (or combinations of such metrics).

[0056] One such metric is a betweenness centrality metric, which quantifies the number of times a node acts as a bridge along the shortest path between two other nodes.

[0057] Another metric is a closeness centrality metric is based on a farness of a node. In the screen connectivity graph 403, a distance metric between a pair of nodes is defined by a length of a shortest path between the nodes of the pair. The farness of a given node is defined as the sum of the distances from the given node to all other nodes in the screen connectivity graph 403. The closeness centrality metric is a reciprocal of the farness.

[0058] Another metric that can be used to measure a centrality of a node is a Hyperlink-Induced Topic Search (HITS) hub metric. The HITS hub metric can be used in link analysis of web pages connected in a network. The HITS hub metric is based on the intuition that a few web pages act as directories or hubs, containing a large number of links to other web pages but little information, whereas web pages linked from these hubs contain actual information. In some implementations of the present disclosure, the HITS hub metric can be used to produce a hub score in the screen connectivity graph 403. [0059] The process of Fig. 4 includes determining (at 406), based on the values of the metric computed for the nodes of the screen connectivity graph 403, which of the nodes correspond to navigation hub screens. For example, screens with metric values greater than a specified threshold can be identified as navigation hub screens.

[0060] Fig. 5 is a graph illustrating example metric values for the six screens (S1 , S2, S3, S4, S5, and S6) with the highest metric values. Curve 502 represents the closeness centrality metric values for screens S1 -S6, curve 504 represents the HITS hub metric values for screens S1 -S6, and curve 506 represents the betweenness centrality metric values for screens S1 -S6.

[0061 ] In the example graph of Fig. 5, screens S1 -S6 can be sorted on the x axis in decreasing order of their betweenness centrality metric values. Among these screens, S1 (e.g. Menu screen 308 in Fig. 3) and S2 (Main Activity screen 302 in Fig. 3) are the navigation hub screens. In Fig. 4, each of the different metrics consistently identifies screens S1 and S2 as the top candidates for navigation hub screens.

[0062] In an example, screens with betweenness centrality metric values greater than a specified threshold, e.g. 0.1 (or some other value) can be selected as navigation hub screens. In the example of Fig. 5, screens with betweenness centrality metric values greater than 0.1 are screens S1 and S2.

[0063] The reason that screen S1 (e.g. Menu screen) is consistently scored higher than screen S2 (e.g. Main Activity screen) is because the Menu screen can be invoked from any screen in the application and thus reachable in a single click. In contrast, the Main Activity screen is the first screen upon application launch, and as a result, the Main Activity screen can only be reached from other screens by a process of clicking the back button multiple times.

[0064] Task Beginner Screen Identification [0065] In cases where the event data acquired by the monitoring agents 1 10 of Fig. 1 is without noise, then detection of task beginner screens can be relatively simple. Specifically, a screen reachable from a navigation hub screen in one hop between respective nodes in the in the screen connectivity graph 403 can be identified as a task beginner screen. A first screen is reachable from a second screen in one hop if the respective nodes representing the first screen and the second screen in the screen connectivity graph 403 are directly connected without another node in between.

[0066] Noise in the event data can be caused by several factors. For example, faulty operation of a monitoring agent 1 10 or faulty operation of the Ul of the application may cause a transition between screens to occur that should not have occurred, which can lead to addition of a false edge in the screen connectivity graph 403.

[0067] In some implementations, to address the foregoing issue, a weight can be assigned to each edge in the screen connectivity graph 403 between a node representing a navigation hub screen (hereinafter referred to as a "navigation hub node") and another node. The weight assigned to a given edge can be based on the number of transitions observed between a navigation hub screen and a second screen represented by the nodes connected by the given edge. A higher weight is assigned in response to observing a larger number of transitions over the given edge. An edge is filtered out if the weight assigned to the edge is less than a threshold. Any screen connected to a navigation hub screen by an edge that has been filtered out is then not identified as a task beginner screen. However, this approach can result in removal of true edges in some cases, such as in cases where transitions between certain screens are associated with tasks that are rarely taken by users.

[0068] In alternative implementations, to avoid filtering out true edges in the screen connectivity graph 403, transition scores can be assigned to transitions between navigation hub screens and other screens, with such transition scores used to identify task beginner screens from among the other screens. [0069] Fig. 6 is a flow diagram of an example process of identifying a task beginner screen using transition scores, in accordance with some implementations. The example process of Fig. 6 can be performed by the analytics engine 108, for example. The process of Fig. 6 includes identifying (at 602), in event data, transitions between navigation hub screens and other screens.

[0070] A user can cause transition from a navigation hub screen to another screen by performing an action, such as a button click or other action. On a given screen, a user may perform several possible actions, each action uniquely identified by a corresponding action ID. Assume a user executes action a on a navigation hub screen H, which causes the application to transition to screen S.

[0071 ] The process of Fig. 6 includes computing (at 604) a count of a number of such transitions (H to S) in the event data collected by the monitoring agent(s) 1 10, where the computed count can be represented as n(H, a, S). Due to faulty operation or errors, not all actions a in the navigation hub screen H lead to screen S. For example, if a user provides an invalid input in the navigation hub screen H, the application may transition to an error screen E, or may not transition but display an error message in the event data stream.

[0072] The process of Fig. 6 also includes computing (at 606) another count, represented as n(H, a), which is the number of times users executed action a on the navigation hub screen H. Note that the count n(H, a) is independent of whether or not the action a in the navigation hub screen H resulted in a transition to screen S. Generally, n(H, a, S) < n(H, a).

[0073] The process of Fig. 6 includes computing (at 608) a transition score, T(H, a, S), that corresponds to an action a performed on the navigation hub screen H that normally should result in transition to screen S. This transition score can be computed as

_T i u \ n(H,a,S)

n(H,a) [0074] According to the above equation, the transition score for a given screen S is computed based on a ratio between a number of times that a given user action taken with respect to the navigation hub screen resulted in a transition to the given screen S, and a number of times that a given user action was taken with respect to the navigation hub screen.

[0075] The process of Fig. 6 includes selecting (at 610) screen 5 as a task beginner screen based on the transition score and the count n(H, a) . More specifically, the screen S is selected as a task beginner screen if T(H, a, S)≥δ and n(H, a) ≥ 2 . The value of δ is a predefined value that can be empirically set.

[0076] The process of Fig. 6 can be repeated for other transitions between navigation hub screens and other screens. The approach of Fig. 6 provides a relatively reliable way of identifying task beginner screens, including in cases where screens are infrequently visited.

[0077] Segmentation

[0078] After identification of navigation hub screens and task beginner screens, the analytics engine 108 can then process the event data to identify task flows as discussed above. The identification of task flows can be performed by segmenting a user session into segments, where each segment of the user session corresponds to a respective task (or multiple respective tasks). Within the event data received by the analytics engine 108, there are two special user actions: (1 ) application launch (when the application is launched), and (2) application exit (when the user exits the application, such as by closing the application or selecting a control element in a screen of the application indicating that a session has been completed).

[0079] A user session refers to a sequence of actions taken by a particular user between the application launch and the application exit. The segmentation process according to some implementations is based on the notion that upon completion of any task, a user may perform one of the following activities:

(A.1 ) access a navigation hub screen to start a new task; (A.2) access a task beginner screen without going through a navigation hub screen (this can occur if the user invoked a shortcut to switch between the tasks); and

(A.3) exit the application.

[0080] An example segmentation pseudocode to extract segments of a user session is presented below.

Segmentation Pseudocode

1 : Input: User Session Sequence {(t _u ^ Sj) (1 < i≤ L)}; navigation hub screen set HS; task beginner screen set B.

2: Output: Task flow segments

3: Let tracingHub=false, tracingBeginner=false;

4: Let Sequence SEG={};

5: for i = 1 to L do

6: if Si ε HS then

7: if not tracingHub and \SEG\≥ 1 then

8: Add SEG to the processing queue; SEG={};

9: end if

10: Add (t _it a _it Si) to SEG;

1 1 : tracingHub =true; tracing Beginner=i a\se;

12: else

13: if si E B then

14: if not (tracingHub or tracing Beginner) and \SEG \≥ 1

then

15: Add SEG to the processing queue; SEG={};

16: end if

17: Add (t^ a^ Si) to SEG;

18: tracing Hub=i a\se; tracing Beginner=true;

19: else

20: Add (t _it a _it Si) to SEG; 21 : tracing Hub=false; tracing Beginner=f a\se;

22: if i == L then

23: Add 5£ ^" G to the processing queue; 5£ ^" G={};

24: end if

25: end if

26: end if

27: end for

[0081 ] The following refers to the foregoing example segmentation pseudocode.

[0082] At line 1 , the input includes User Session Sequence {(ΐι, αι, Ξι) (1 < i≤ L)}, navigation hub screen set HS, and task beginner screen set B. The User Session Sequence can include a stream of event records (L event records in total), such as depicted in Table 1 above. The navigation hub screen set HS includes one or multiple navigation hub screens identified by the example process of Fig. 4, while the task beginner screen set B includes the task beginner screens identified by the example process of Fig. 6.

[0083] At line 2 of the segmentation pseudocode, the output is specified as the task flow segments produced by segmenting the user session (represented by {(ti, ai, Si (1 < i≤ L)}) by the segmentation pseudocode.

[0084] The segmentation pseudocode parses the user session in sequential order (from event record 1 to event record L at line 5), and builds up a potential candidate segment SEG to publish. Upon observing a screen s _{i t} the segmentation pseudocode determines if one of the three conditions corresponding to activities (A.1 ), (A.2), and (A.3) listed above, that mark the end of a task is met. Line 6 of the segmentation pseudocode performs a check to determine if the screen s _t is a navigation hub screen, which corresponds to activity (A.1 ). Line 13 of the

segmentation pseudocode performs a check to determine if the screen s _t is a task beginner screen, which corresponds to activity (A.2). Line 22 of the segmentation pseudocode performs a check for the end of the user session, which corresponds to activity (A.3).

[0085] In the segmentation pseudocode, the parameter tracingHub if true indicates that a navigation hub screen has been encountered, and the parameter tracingBeginner if true indicates that a task beginner screen has been encountered.

[0086] Lines 19-23 of the segmentation pseudocode are performed if an observed screen is not a navigation hub screen or a task beginner screen. Lines 19-23 add an event record (t ₀ α^) to the currently processed segment SEG.

[0087] If a navigation hub screen or a task beginner screen is detected (at lines 6 and 13, respectively), then the segmentation pseudocode outputs the segment SEG (using the code "Add SEG to the processing queue" at lines 8 and 15, respectively, wherein "processing queue" is an output queue to which the segments can be added), and starts again with a new candidate segment SEG (by setting SEG={} at lines 8 and 15, respectively).

[0088] In the example of Fig. 3, activity (A.1 ) occurs when the Menu screen 308 is invoked by a user; this activity causes the segmentation pseudocode to output a sub-sequence from the Main Activity screen 302 to the Flight List screen 306 as a first segment.

[0089] In the example of Fig. 3, activity (A.2) occurs when the Hotel Search screen 320 is invoked by the user; this activity causes the segmentation pseudocode to output a sub-sequence from the Menu screen 308 to the Baggage Information screen 318 as a second segment. Note, however, as explained above in connection with Fig. 3, this second segment includes two tasks, task (2) to read passenger details for a booked flight, and task (3) to read baggage information for a booked flight.

[0090] Finally, activity (A.3) is reached at the end when the algorithm outputs the sub-sequence from the Hotel Search screen 320 to the Confirmation screen 328 as a third segment. The first and third segments each has only one task flow, whereas the second segment has two task flows. The two task flows in the second segment occurs because the user switches from task (2) to task (3) without going through a navigation hub screen or task beginner screen.

[0091 ] Maximal Forward Flow Extraction

[0092] As noted above, a segment identified by the segmentation process can include more than one task flow. Fig. 7 shows an example of the screens (the screens are assigned reference numerals of corresponding screens appearing in Fig. 3) in the second segment extracted above that has two task flows. The two task flows of the second segment shares a common path, Menu→ Booked Flights→ Flight Details.

[0093] In Fig. 7, the first path for the first task is as follows: Menu→ Booked Flights→ Flight Details→Passenger Details. However, to perform the second task, the user retraced backwardly to a screen in this example the Flight Details screen 312) in the common path, and then moved forward towards the another screen, the Baggage Information screen 318, for the second task. Thus, the complete task flow for the second task can be obtained by concatenating the next screen, the Baggage Information screen 318, with the shared common path. The path of the second task is thus Menu→ Booked Flights→ Flight Details→Baggage Information.

[0094] The process of deriving the full path for the second task due to user retracing backwardly through at least one screen after the user is done with the first task is referred to as a maximal forward flow extraction process. In response to detecting that the user has retraced to a previous screen, the maximal forward flow extraction process removes all nodes visited along the reverse path (the Passenger Details screen 314 in the example of Fig. 7). The maximal forward flow extraction process then proceeds to find the next maximal forward path for the second task, which is Menu→ Booked Flights→ Flight Details→Baggage Information. [0095] Data Model

[0096] As discussed above in connection with Fig. 1 , information derived from the extracted task flows 124 can be accessed by the application developer station 126, such as by the application developer station 126 submitting queries to the analytics server 102. Over time, the number of extracted task flows 124 can increase due to an increase in the amount of event data collected by the monitoring agents 1 10. In some cases, there can be multiple task flow instances for the same task, where the multiple task flow instances for the same task can have the same collection of user actions, but can differ in the order the user actions were performed. For example, if a screen in the task has multiple input fields (e.g. origin city field and destination city field of a Flight Search screen) in which corresponding input text is to be provided, a user may provide the input text into the multiple input fields in arbitrary order. However, the task execution path for the multiple task flow instances of the same task remains the same. Thus, the analytics engine 108 can group task flow instances that share the same task execution path, and compute aggregate statistics on the grouped task flow instances. When a new task flow instance for a given task is observed, the analytics engine 108 finds the statistics for the

corresponding task execution path, and updates such statistics. To make the process of finding the corresponding task execution path for a new task flow instance more efficient, task execution paths can be stored in a prefix-tree (also referred to as a Trie data structure).

[0097] A trie is an ordered tree data structure that is includes a tree-based arrangement of nodes. Unlike a binary search tree, no node in the trie stores the key associated with that node; instead, the position of the node in the trie defines the key with which the node is associated. The key can be the string that represents a respective task execution path. The descendants of a node have a

common prefix of the string associated with that node, and the root is associated with an empty string. Values associated with intermediate nodes of the trie can be stored by leaf nodes. In the present disclosure, the values are the statistics associated with a respective task execution path, and the statistics can be stored at a leaf node of the trie.

[0098] Additionally, in some examples, each intermediate node in the trie can store (a) a number of leaf nodes reachable from the intermediate node, and (b) a number of task flow instances captured by the intermediate node. The number of leaf nodes reachable from the intermediate node gives the number of distinct paths taken by users via the intermediate node, and the number of task flow instances captured by the intermediate node gives the popularity of the intermediate node.

[0099] In other examples, other data structures can be used to store task execution paths and respective statistics.

[00100] Example Systems

[00101 ] Fig. 8 is a block diagram of an example system 800 according to some implementations. The system 800 can be implemented as a computer or a distributed arrangement of computers. The system 800 includes a processor (or multiple processors) 802. A processor can include a microprocessor, a core of a multi-core microprocessor, a microcontroller, a PGA, an ASIC device, or any other hardware processing circuit.

[00102] The system 800 further includes a non-transitory storage medium (or storage media) 804 that stores machine-readable instructions executable on the processor(s) 802. The machine-readable instructions include instructions that are executable to perform various tasks of the analytics engine 108 according to the present disclosure including any of those discussed above. In some

implementations, the machine-readable instructions include instructions 806 to receive event data of events representing a collection of user actions made with respect to screens presented by an application, instructions 808 to identify, based on the event data, a navigation hub screen, instructions 810 to identify a task beginner screen, and instructions 812 to extract, based on transitions among screens including the navigation hub screen and the task beginner screen, task flows, each task flow of the task flows including respective multiple user actions of the collection of user actions.

[00103] Fig. 9 is a block diagram of an example non-transitory machine-readable or computer-readable storage medium 900 according to some implementations. The storage medium 900 stores machine-readable instructions that are executable in a system to perform various tasks of the analytics engine 108, including any of those discussed further above. The machine-readable instructions stored in the storage medium 900 can include instructions 902 to receive event data of events

representing a collection of user actions made with respect to a plurality of screens presented by an application, instructions 904 to identify a session (e.g. a user session) that begins at a launch of the application and ends at an exit of the application, and instructions 906 to segment the session into segments based on identifying invocations of at least one starter screen. A first segment of the segments includes a first task flow including multiple user actions of the collection of user actions, and a second segment of the segments includes a second task flow including multiple user actions of the collection of user actions.

[00104] Each of the storage media 804 and 900 can include one or multiple different forms of memory including semiconductor memory devices such as dynamic or static random access memories (DRAMs or SRAMs), erasable and programmable read-only memories (EPROMs), electrically erasable and

programmable read-only memories (EEPROMs) and flash memories; magnetic disks such as fixed, floppy and removable disks; other magnetic media including tape; optical media such as compact disks (CDs) or digital video disks (DVDs); or other types of storage devices. Note that the instructions discussed above can be provided on one computer-readable or machine-readable storage medium, or alternatively, can be provided on multiple computer-readable or machine-readable storage media distributed in a large system having possibly plural nodes. Such computer-readable or machine-readable storage medium or media is (are) considered to be part of an article (or article of manufacture). An article or article of manufacture can refer to any manufactured single component or multiple components. The storage medium or media can be located either in the machine running the machine-readable instructions, or located at a remote site from which machine-readable instructions can be downloaded over a network for execution.

[00105] In the foregoing description, numerous details are set forth to provide an understanding of the subject disclosed herein. However, implementations may be practiced without some of these details. Other implementations may include modifications and variations from the details discussed above. It is intended that the appended claims cover such modifications and variations.