SYSTEM AND M ETHOD FOR PROVI DI NG A WEB BROWSER ON LIN E USER I NTERFACE WITH HAPTIC FEEDBACK FOR AN AUTOMOTIVE SETTI NG

TECHNICAL FIELD

[0001] The present disclosure belongs to the technical field of automotive passenger interfaces. In particular, the present disclosure relates to a system and method for providing a web browser online interface comprising a display surface with haptic feedback for a user in an autonomous vehicle setting.

BACKGROUND

[0002] Document US2017153702 providing and/or determining haptic feedback may include, responsive to determining that a user is proximate to a haptic feedback device, determining, using a processor, a location of the haptic feedback device, and receiving, using the processor, a plurality of external factors determined from the location of the haptic feedback device, biometric data for the user, and social networking data for the user. The method may also include determining, using the processor, haptic feedback for use by the haptic feedback device by applying the external factors to a haptic feedback template of an object.

[0003] Document US2018046250 describes frame structure with downlink/uplink subframe configuration and channel hopping scheme for unlicensed narrowband Internet-of-Things (loT) systems. An apparatus operable for unlicensed narrowband transmission to support loT service is disclosed. The apparatus includes baseband circuitry to select a transmission channel within an unlicensed narrow band for downlink transmission of a discovery reference signal (DRS), and for channel hopping, to select, according to the DRS, a communication channel within the unlicensed narrow band for downlink data and uplink data. The DRS includes a primary synchronization signal (PSS), a secondary synchronization signal (SSS) and physical broadcast channel (PBCH) content. The baseband circuitry is further to demodulate a received signal received via radio frequency (RF) circuitry through the communication channel over an uplink frame, and to modulate a transmitting signal to be transmitted via the RF circuitry through the communication channel over a downlink frame.

[0004] Document US20200264772 discloses a method for providing haptic feedback to an operator of a touch-sensitive display device. The method includes providing an operator interface to be represented in the display device as image data; the operator interface having at least one button assigned to a function to be controlled, graphically displayed in the operator interface, and graphically delimited from the rest of the operator interface; analysing the image data for a presence and a position of shapes for representing a button; determining an intended display region of a shape, which is to be represented and is identified as a button; representing the image data of the operator interface in the touch-sensitive display device; outputting haptic feedback in response to detection of contact with the touch-sensitive display device in the region of a surface of the touch-sensitive display device which is assigned to a display region of a shape identified as a button.

[0005] These facts are disclosed in order to illustrate the technical problem addressed by the present disclosure.

GENERAL DESCRIPTION

[0006] The present disclosure discloses on making use of a smart surface, composed of an insulative layer, fitted atop a transparent and conductive electrode sheet layer, which, when charged with an electric current and when in contact with the human skin, produces a frictional stimulus by controlling the electric current. This layer is in turn fitted atop a capacitive-based touch sensitive layer, fitted atop a layer of LED lights, which in turn are fitted atop a layer of vibrotactile actuators. The layer of LEDs is optional and when not present the touch-sensitive layer is fitted atop a layer of vibrotactile actuators.

[0007] Recently, some auto manufactures have started to incorporate web browsing applications (apps) in their vehicle's infotainment systems (e.g., Tesla®, n.d.). The integration and use of this kind of app in a vehicle's infotainment system, while the vehicle is being driven, raises serious concerns regarding the risks of driver distraction, especially if the one using the app is the driver himself/herself.

[0008] However, as the automotive industry evolves, it is expected that users will need to spend less time actively driving their vehicles, which will in turn result in them having more free time for themselves during their commutes.

[0009] Most in-vehicle infotainment displays and surfaces are not equipped with haptic feedback capabilities, so the interactions between users and these devices are very basic.

[0010] Haptic feedback in devices and surfaces has commonly been achieved through vibrotactile stimuli, generated from mechanical actuators (e.g., EP3582074A1).

[0011] More recently, technologies have emerged that make use of other actuation types, such as electro vibration, to generate other types of tactile feedback beyond just vibrations (e.g., US10768749B2). The technology for creating richer display and surface interactions is available, by delivering different haptic feedback sensations, such as different texture sensations or different vibrations, to users while they interacted with the display surface with their fingers. However, there is a lack of apps that exploit the capabilities of these interface to the maximum.

[0012] With the expected incoming increase of autonomous vehicles that will become available to the general consumer market, users of such vehicles will find themselves with more free time during their commutes. Users might choose to engage in a variety of activities to make use of this free time, for leisure or work purposes, including accessing the internet for shopping, browsing social media, or looking up information.

[0013] To the best of our knowledge, not many web browsing apps, developed mainly for use inside or alongside a vehicle's infotainment systems, currently exist. For example, the Tesla Web Browser, developed for use on some Tesla Vehicles, allows users to navigate the web using the vehicle's centre stack display (Tesla®, 2018). However, this centre stack display does not provide haptic feedback to users, neither does the web browser application incorporate information and instructions for devices to interpret and deliver haptic feedback sensations to users. Moreover, this Web Browser application possesses several limitations in order to minimize driver distraction during the task of driving. As display surface technology develops, more focus is being given to display devices and surfaces that can provide new and innovative ways of enriching the interactions users have with them, such as generating richer haptic feedback sensations through the use of frictional and vibrotactile stimuli. In level 5 of autonomous vehicles, i.e., in full automation, concerns regarding driver distraction are no longer present, as the task of driving the vehicle is taken up by the vehicle's system, and the driver becomes just another passenger in the vehicle. Therefore, passengers may choose to use the time during their commutes for other tasks, such as reading, watching media, browsing the internet, or online shopping. Therefore, it can be expected that users will pay more attention to the characteristics and quality of their interactions with different elements of the vehicle, such as different surfaces and the central stack display. For this reason, by adding the capability of generating different haptic feedback sensations on these surfaces and display, and developing software that can make use of said feedback sensations, a more enjoyable and pleasurable experience can be delivered to users of these autonomous vehicles.

[0014] The present disclosure can be applied to a train carriage, airplane fuselage or other passenger compartment.

[0015] It is disclosed a web browsing interface, with the aim of it being used primarily in level 5 autonomous vehicle infotainment displays, that makes use of the richer interaction opportunities that some haptic feedback display and surface technologies currently offer, while also planning for what these technologies and other haptic feedback technologies are able to provide users with.

[0016] The present disclosure is employable in level 5 autonomous vehicles.

[0017] This smart surface can be employed either as: a) the interactive surface interface of a display device to which it is physically connect, e.g., the smart surface is part of the device's display; or b) as the interactive surface interface of a display device to which it is not physically connected, e.g., touch surface interface communicating wirelessly with a display device in close proximity.

[0018] The capacitive-based touch sensitive layer can be made of a variety of materials, depending on the intended use case and aesthetic appeal. [0019] In an embodiment, the capacitive-based touch sensitive layer is made of transparent glass - display device; translucent plastic - interactive surface interface for an external device, using the LED lights layer as the display component of said device.

[0020] In this present disclosure, the smart surface can generate both frictional and vibrotactile haptic sensations.

[0021] Frictional haptic sensations are generated thanks to the insulative and conductive layers, which allow for friction between the smart surface and the user's skin to be modulated through electrostatic actuation, while the skin is in motion.

[0022] Vibrotactile haptic sensations, in turn, are generated thanks to the vibrotactile actuators, which can generate vibrations strong enough to be felt through the user's skin, either stationary or in motion, when in direct contact with the smart surface. When the smart surface is used as the interactive interface for an external device, haptic feedback sensations, caused by the external device's software, can be generated on it, as long as the haptic actuation technology that is present on the smart surface is both capable of interpreting the software's information and of generating such sensations.

[0023] For the present disclosure being discussed in this document, this smart surface is to be fitted on at least two regions inside the vehicle: as the display surface of the central stack display (these can provide vibrotactile feedback, while this application would make use of both frictional and vibrotactile feedback), where it can serve as said device's interactive display surface interface (kind of like how a laptop's mousepad controls the mouse shown on the screen, interacting with the interactive surface of the armrest would control what is shown on the center stack display device); and on a portion of the surface of the seat's armrest, where it can act as an external interactive (Both communicate through the vehicle's internal infotainment software system, which is the main controller of both ends) surface interface for the center stack display device. In both cases, the smart surface is connected to the internal electronics of the vehicle's systems, be it through the centre stack display device, which is directly connected and controls the vehicle's infotainment system, or, in the case of the armrest's smart surface, directly connected to the vehicle's infotainment system. In this way, commands and interactions inputted through the armrest's smart surface, can be interpreted, and visualized on the centre stack display. On the other hand, information displayed on the centre stack display that possesses commands for haptic feedback sensations can also be sent to and actuated on the armrest's smart surface surface interface for the centre stack display device.

[0024] The haptic feedback web browser app can be installed and accessed through several different devices, such as the vehicle's infotainment system or the user's personal smartphone. Additionally, in order to access and display web content, the device has access to the Internet and is able to connect to it.

[0025] The interaction between user and app is more fruitful when done through the previously disclosed smart surface, especially when said interaction occurs inside the vehicle, during the user's commute.

[0026] The herein described haptic feedback sensations make use of frictional and/or vibrotactile stimuli, which can be generated through electrostatic and vibrotactile actuator, respectively.

[0027] When the app is accessed through a device without a surface that can produce one or both of the intended haptic feedback stimulus types, an adequate external interactive surface interface can be used to generate said stimuli. For this, said surface should be somehow able to connect to the device, either through a wired or a wireless connection, and should also possess either one or both types of actuators necessary to generate the intended hapticfeedback stimuli, in orderto receive, interpret, and be able to generate the intended haptic feedback sensations.

[0028] Although one of the main features of this app is the employment of haptic feedback sensations during the interactions between it and the user, there are no impediments regarding installing and using this app on devices without actuators that can produce the stimuli types from which this app benefits, as long as users keep in mind that the full intended experience cannot be achieved in said devices.

[0029] There is also no impediment in the user choosing to use an external interactive surface interface to interact with the app when displayed on a device which also possesses a screen surface with haptic feedback capabilities. For example, users might choose to interact with the app through the armrest' smart surface, while leaning back on their seat, for extra comfort, while others might prefer to interact with it through the centre stack display itself, interacting "directly" with the visual content displayed on the screen.

[0030] In one embodiment, the visual content of the app is always displayed on the main interface screen.

[0031] In one embodiment, the app can access and present web content to users, such as text and images. Wherein said web content is linked with a corresponding haptic feedback sensation, said feedback is also generated on the device's smart surface so that users can feel it, provided that the actuators present on said smart surface can generate the intended feedback sensations (e.g., textures).

[0032] In one embodiment, the app is displayed on the center stack display device.

[0033] In one embodiment, the haptic feedback sensations can be felt during user tactile interactions with either the display device's smart surface, or the armrest' smart surface.

[0034] The haptic feedback information associated with each web content is to be designed and/or provided by the address hosting said web content. This information is then downloaded by the app, which assesses if the connected devices/surfaces can reproduce the haptic feedback sensations. If yes, the haptic feedback sensations are generated either on all connected and available surfaces, or on those surfaces which the user has previously selected. If not, haptic feedback sensations are not generated.

[0035] It is disclosed a system comprising a web browser online user interface with haptic feedback for an automotive setting, comprising: a haptic surface comprising actuators for exhibiting tactile textures, comprising: a capacitive-based touch sensitive layer, a frictional haptic layer comprising an insulative sheet for user touch and a transparent conductive electrode film for providing frictional haptic sensations, and a vibrotactile haptic layer for providing vibrotactile haptic sensations; an electronic data processor configured to: provide the web browser online user interface on a display for displaying images of patterns or objects; display an image on the web browser interface on said display, said image comprising a pattern or object; and drive the haptic layers to exhibit a tactile texture when the user touches the image of the pattern or object, wherein the texture mimics tactile sensations corresponding to the pattern or object which image is being displayed.

[0036] In an embodiment, the system further can comprise said display for displaying images of patterns or objects, wherein the haptic surface is superimposed on the display.

[0037] In an embodiment, further comprising a seat armrest, wherein the haptic surface is superimposed on the display which is arranged on said armrest.

[0038] In an embodiment, the haptic surface is arranged to communicate wirelessly with the electronic data processor.

[0039] In an embodiment, said display is arranged to communicate wirelessly with the electronic data processor.

[0040] In an embodiment, the automotive setting is an autonomous vehicle and the display is a centre stack display of the autonomous vehicle.

[0041] In an embodiment, the automotive setting is an autonomous vehicle and comprising a secondary display which is a centre stack display of the autonomous vehicle, wherein the electronic data processor is further configured to: provide the web browser online user interface on the secondary display for displaying images of patterns or objects; display the image on the web browser interface on said secondary display.

[0042] In an embodiment, the secondary display comprises:

[0043] a haptic surface comprising actuators for exhibiting tactile textures and superimposed on said secondary display, comprising: a capacitive-based touch sensitive layer, a frictional haptic layer comprising an insulative sheet for user touch and a transparent conductive electrode film for providing frictional haptic sensations, and a vibrotactile haptic layer for providing vibrotactile haptic sensations; wherein the electronic data processor is further configured to: drive the haptic layers of the secondary display to exhibit a tactile texture when the user touches the image of the pattern or object, wherein the texture mimics tactile sensations corresponding to the pattern or object which image is being displayed on said secondary display. [0044] In an embodiment, said two displays are arranged to communicate wirelessly with the electronic data processor.

[0045] In an embodiment, the electronic data processor is further configured to select either a frictional or a vibrotactile haptic feedback as a function of the pattern or object to be displayed.

[0046] In an embodiment, the capacitive-based touch sensitive layer is made of translucent plastic comprising LEDs underneath it.

[0047] It is also disclosed a method of operating a system according to any of the embodiments, comprising using said electronic data processor for: provide the web browser online user interface on a display for displaying images of patterns or objects;

[0048] display an image on the web browser interface on said display, said image comprising a pattern or object; and drive the haptic layers to exhibit a tactile texture when the user touches the image of the pattern or object, wherein the texture mimics tactile sensations corresponding to the pattern or object which image is being displayed.

BRI EF DESCRI PTION OF THE DRAWI NGS

[0049] The following figures provide preferred embodiments for illustrating the disclosure and should not be seen as limiting the scope of invention.

[0050] Figure 1: Schematic representation of an embodiment of a home/quick access page layout.

[0051] Figure 2: Schematic representation of an embodiment of a clothing shop website layout.

[0052] Figure 3: Schematic representation of an embodiment of a furniture shop website layout.

[0053] Figure 4: Schematic representation of an embodiment of an interface with a capacitive, haptic feedback on armrest and central display. DETAILED DESCRI PTION

[0054] The present disclosure relates to a system comprising a web browser online user interface with haptic feedback for an automotive setting, comprising: a haptic surface comprising actuators for exhibiting tactile textures, comprising: a capacitive-based touch sensitive layer, a frictional haptic layer comprising an insulative sheet for user touch and a transparent conductive electrode film for providing frictional haptic sensations, and a vibrotactile haptic layer for providing vibrotactile haptic sensations; an electronic data processor configured to provide the web browser online user interface on a display for displaying images of patterns or objects; display an image on the web browser interface on said display, said image comprising a pattern or object; and drive the haptic layers to exhibit a tactile texture when the user touches the image of the pattern or object, wherein the texture mimics tactile sensations corresponding to the pattern or object which image is being displayed. The disclosure also relates to the operation method thereof.

[0055] Figure 1 shows a schematic representation of an embodiment of a home/quick access page layout search or URL navigation field, which provides a frictional stimulus sensation when the user's skin interacts with its location on the screen; 102 represents an image preview of a bookmarked webpage, which provides a frictional stimulus sensation when the user's skin interacts with its location on the screen; 103 represents a button to add a new bookmark to the home/quick access page layout, which provides a frictional stimulus sensation when the user's skin interacts with its location on the screen.

[0056] Figure 2 shows a schematic representation of an embodiment of a clothing store's item selection webpage, where: 101 represents a preview window showing how the clothing item looks like with the selected fabric material, which provides a frictional stimulus sensation when the user's skin interacts with its location on the screen; 102 represents the selection button for users to select which material they want to buy; 103 represents a zoomed in area of the selected fabric material is previewed, which provides a frictional stimulus sensation when the user's skin interacts with its location on the screen. [0057] Figure 3 shows a schematic representation of an embodiment of a furniture store's item selection webpage, where: 101 represents a preview window showing how the furniture item looks like with the selected material, which provides a frictional stimulus sensation when the user's skin interacts with its location on the screen; 102 represents the selection button for users to select which material they want to buy; 103 represents a zoomed in area of the selected material is previewed, which provides a frictional stimulus sensation when the user's skin interacts with its location on the screen.

[0058] Figure 4 shows a schematic representation of an embodiment of a capacitive haptic surface on armrest, where: 11 represents a passenger, 12 represents the seat, 13 represents the armrest with the capacitive, haptic surface, and 14 represents the center stack display that the capacitive, haptic surface of the armrest is in contact with.

[0059] As illustrated in Figure 1, a system can comprise a web browser online user interface with haptic feedback for an automotive setting, comprising: a haptic surface comprising actuators for exhibiting tactile textures, comprising: a capacitive-based touch sensitive layer, a frictional haptic layer comprising an insulative sheet for user touch and a transparent conductive electrode film for providing frictional haptic sensations, and a vibrotactile haptic layer for providing vibrotactile haptic sensations; an electronic data processor configured to: provide the web browser online user interface on a display for displaying images of patterns or objects; display an image on the web browser interface on said display, said image comprising a pattern or object; and drive the haptic layers to exhibit a tactile texture when the user touches the image of the pattern or object, wherein the texture mimics tactile sensations corresponding to the pattern or object which image is being displayed.

[0060] As illustrated in Figure 2 a schematic representation of an embodiment of a clothing shop website layout.

[0061] As illustrated in Figure 3 a schematic representation of an embodiment of a furniture shop website layout.

[0062] In an embodiment and according to the technical field literature, examples of textures that participants subjectively report feeling during interaction with electrode devices include: wood, leather, paper, painted wall, rubber, and viscous liquid (Bau, Olivier, and Ivan Poupyrev. "REVEL: tactile feedback technology for augmented reality.

ACM Transactions on Graphics (TOG) 31.4 (2012): 1-11).

[0063] Through the use of an electrode, it is possible to modify the friction that exists between the finger and an actuated surface, when there is movement of one or both parts. Through friction, we can manipulate, for example, the feeling of how smooth or rough a surface is. This, however, is normally limited in two respects, namely how smooth the surface on which the finger slides is (because electrode only creates friction, does not remove it, so the least possible friction to create is the natural friction between the surface and the finger); and, also, how much electrical charge can be given to the surface actuated (the higher the electrical charge, the greater the friction generated between the finger and the surface), up to a certain limit, which is depending on the structure of the device itself and its electric system.

[0064] The term "comprising" whenever used in this document is intended to indicate the presence of stated features, integers, steps, components, but not to preclude the presence or addition of one or more other features, integers, steps, components or groups thereof.

[0065] The disclosure should not be seen in any way restricted to the embodiments described and a person with ordinary skill in the art will foresee many possibilities to modifications thereof. The above-described embodiments are combinable. The following claims further set out particular embodiments of the disclosure.

[0066] References

1. KHOSHKAVA, V., CRUZ-HERNANDEZ, J. M., & SHAH, K. (2019). HAPTIC ACTUATOR ASSEMBLY WITH A SPRING PRE-LOAD DEVICE (Patent No. EP3582074A1).

2. Olley, M. F. D., Peshkin, M. A., & Colgate, J. E. (2020). Electronic controller haptic display with simultaneous sensing and actuation (Patent No. US10768749B2).

3. Tesla ©. (n.d.). Software Version 9.0. Retrieved February 17, 2021, from https://www.tesla.com/support/softwarev9?redirect=no.

4. Tesla®. (2018). Introducing Software Version 9.0. https://www.tesla.com/blog/introducing-software-version- 9?redirect=no%3Futm_campaign%3DV9&utm_source=Twitter& ;utm_medium=soc ial.