FOR IMAGE OR VIDEO DATA

TECHNICAL FIELD

[0001] The present disclosure relates to compressing or processing image or video data using devices having rechargeable battery. More particularly, the disclosure relates to method for saving battery and extending the battery life, as well as the service life of the battery when the device using rechargeable battery needs to compress or process image or video data.

BACKGROUND OF THE INVENTION

[0002] In 2018, there are 4 billion smart phones in the world. In 2020, there will be 6.1 billion people using smart phones. It is predicted that in the fourth quarter of 2018, each smart phone will store approximately 60 GB of data on average, most of which is image and video data (https://www.counterpointresearch.com/average-smartphone-nan d-storage-capacitv- will-ton- 60gb-bv-20l8/I.

[0003] Devices using rechargeable battery such as smart phone, camera, video recorder, laptop usually collect and store a large amount of external data in the devices, specially image and video data. Images and videos always need to be stored and transmitted in the form of lossy compression. Even if being executed by a specialized integrated circuit (IC) or executed by a software, data compression is a process that consumes a large amount of processing capacity and electricity to reduce the data size, thereby reducing the storage and transmission costs. The problem is that the devices using rechargeable battery usually have a battery life shorter than the usage demand, therefore the hardware and software designer usually selects the compression method that is optimal for saving battery instead of optimal for reducing the data size. On the other hand, if the compression method is too complicated, the electronic device will consume a large amount of energy and generate an amount of heat larger than the amount of heat that can be discharged to the environment, and thus causing the device using rechargeable battery to stop operating to wait for the temperature to decrease or to face an explosion risk. Because of these problems regarding battery and temperature, the existing devices using rechargeable battery are usually provided with compression technologies using IC but excluding the best compression technologies that are battery-consuming. This causes a large data size of video being stored in the electronic device, the video is usually 2 to 3 times larger than if the video is compressed using the best available compression technology.

[0004] Before discussing the existing solutions to save battery and extend the battery life as well as the service life of the battery when compressing or processing image or video data, some related technical definitions are defined in details as follows.

[0005] Battery life is the time period that the device can operate, since the charging stops until the battery runs out. Assuming that the energy consumed by the device remains stably unchanged during the entire process. The unit of the battery life may be a time unit such as second, hour, or percentage, which is the actual battery life divided by the maximum battery life of the device.

[0006] Acceptable temperature threshold is a temperature threshold predefined by the device or set up by the user. One option to set up this temperature threshold is to select the value which guarantees that the device is safe and does not explode due to overheat; another option is to select the value which guarantees not to cause an uncomfortable feeling or an explosion concern as the user holds the device while the device is performing the data processing method; another option is to select the value which guarantees not to harm the service life of the battery because the service life usually reduces significantly when the battery is overheated. [0007] Limiting data processing speed is a method that causes the processing operations to run slower than the capacity of the device such that the system meets certain requirements. Limiting data processing speed can be achieved using different implementations, for example, using functionalities supported by the operating system, or hibernating the entire device for a time period and then resuming the operations of the device.

[0008] Typically, a video after being captured by a sensor is compressed using a popular compression standard such as H.264/AVC, H.265/HEVC, etc. Fundamental concept of these compression standards is to divide the video into groups of frames. Each frame is then divided into macroblocks (e.g., 16x16 macroblock in H.264/AVC). The process of compressing each macroblock is performed by prediction, residual quantization, and entropy compression.

[0009] H.264, which refers to MPEG-4 Part 10 AVC (Advance Video Coding) and is usually abbreviated as MPEG-4 AVC/H.264 is a potential compression format that can improve the image compression efficiency higher than other common compression formats such as MPEG-4 or MPEG-2.

[0010] Being first introduced in 2004, H.264 standard has become one of the most popular video compression standards in recording, compressing, and transmitting images with high display quality and reduced bandwidth.

[0011] H.264 standard is now popularly used in applications such as Blu-ray disc.

Camera recorder, video recorder, set-top-box, etc., are all capable of decoding H.264 format. This standard is also used in video streaming services (e.g., Youtube, Vimeo, iTunes Store, etc.), web software (e.g., Adobe Flash, Microsoft Silverlight, HTML5, etc.).

[0012] H.265 standard is a potential alternative compression standard for the H.264 standard which is very popular today. H.265 standard is capable of compressing the data 2 times more than other existing compression standards while maintaining the image quality. This standard can support the resolution up to 8192 x 4320, including 8k Ultra HD quality.

[0013] Existing solutions include:

[0014] U.S. Patent No. US7911509B2 discloses a solution for selecting adaptive video encode scaling in which the processing logic circuit selects the video encode scaling based on a condition of the memory, a condition of the power supply (battery), or processing logic.

[0015] U.S. Patent Publication No. US20080084491A1 discloses a solution for selecting video processing scaling based on a power savings factor, in particular, adjusting complexity of video encoding based on the power savings factor.

[0016] U.S. Patent Publication No. US20060039469A1 discloses a solution for scalable video compression based on remaining battery capacity.

[0017] U.S. Patent Publication No. US20060215755A1 discloses video encoding method and system for battery-powered apparatus in which picture type, picture size, and picture rate are determined based on the power level of the battery.

[0018] U.S. Patent Publication No. US20120083258A1 discloses a solution to manage the processing of the mobile device based on available power resources, in which a battery threshold calculator provides a notification of a battery charge threshold sufficient for processing a scheduled program that has a certain processing complexity.

[0019] U.S. Patent Publication No. US20150181208A1 discloses a method for managing thermal and power of a device for video encoding, which includes the step of adjusting encoding parameters based on operating characteristics of the components of the electronic device and based on video quality. [0020] None of these documents discloses a solution for processing data using the best technologies although they may have higher processing complexity in order to maximally reduce the data size and completely avoid reducing the battery life.

DISCLOSURE OF THE INVENTION

[0021] The purpose of the invention is to provide a technology for devices using rechargeable battery to apply data processing method that has highest efficiency while still guarantee that the battery life is comparable to existing processing methods.

[0022] According to one embodiment, the invention proposes a non-real-time data processing method for image or video data on a device using rechargeable battery, the method includes the following steps:

[0023] receiving data that can be compressed by the device using rechargeable battery;

[0024] processing the received data only during a time period in which the device is charged by an external source; and

[0025] limiting a data processing speed such that a temperature of the device using rechargeable battery does not exceed an acceptable temperature threshold.

[0026] According to one particular embodiment of the invention, the data processing method is a data compressing method, and data processing is data compression, compressed data is an image or a video that was previously compressed but is currently decompressed and then recompressed using a method that is more optimal in terms of data size.

[0027] According to one particular embodiment of the invention, the method further includes a step of processing the data only when a screen is off.

[0028] According to one particular embodiment of the invention, the method further includes a step of processing the data only when a user does not input data into the device during a particular time period. The data input method may be typing on a key, touching sensors or buttons. Example time period can be any value from 1 minute to 1 hour.

[0029] According to one particular embodiment of the invention, the method further includes a step of limiting the data processing speed such that the processing operation does not impact other operations of the device, e.g., synchronizing data from the device to the Internet.

[0030] According to one particular aspect of this embodiment, the method further includes a step of processing the data only when the processing operation does not occupy Random Access Memory (RAM) of the running applications.

[0031] According to one particular embodiment of the invention, the method further includes a step of not processing the data when the device is charged by Portable Power Banks.

[0032] According to one particular embodiment of the invention, the method further includes a step of limiting the data processing speed such that the energy required for processing is sufficiently provided by the external source, therefore the data processing operation does not reduce battery life.

[0033] According to one particular embodiment of the invention, the method further includes a step of limiting the data processing speed such that the battery life is always above a value predefined by the device, this value may be redefined by the user. For example, this value may be 90% or 100%.

[0034] According to another embodiment of the invention, the data processing method is an image recognition method to extract useful information from the data, wherein data processing is recognizing image to extract useful information from the data, or classifying the data to enable a user to mine the data more efficiently. [0035] According to another embodiment of the invention, the data processing method is a data analyzing method, wherein analyzing data is to suggest the user to make a data processing decision such as backup or delete redundant data.

[0036] Additionally, according to another embodiment, the invention proposes a device using rechargeable battery (1) including at least the following components: a rechargeable battery (11), an image processing IC or chip (12), a screen (13), a microcontroller unit (MCU) (14), a camera (15), a microphone (16), a speaker (17), a temperature sensor (18), an input component (19), other sensors (110), and memory (111). The MCU (14) of the device using rechargeable battery (1) is configured to control the components of the device using rechargeable battery (1) to perform the non-real-time data processing method of the invention according to one of the embodiments and aspects discussed above. Examples of the device using rechargeable battery (1) may be mobile phone, smart phone, tablet, even laptop using rechargeable battery, etc.

[0037] Additionally, according to another embodiment, the invention also proposes a computer program product or a computer readable medium including a computer program, the computer program includes instructions that cause suitable hardware components to perform the non-real-time data processing method according to the invention.

[0038] By applying the solution according to the invention, when compressing data, the devices using rechargeable battery are capable of compressing image or video using the best technologies that maximally reduce the data size and completely avoid reducing the battery life as compared to existing methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Aspects and advantages of the invention are described and reference to the following drawings:

[0040] Figure 1 is a flowchart of a non-real-time data processing method for image or video data on a device using rechargeable battery according to one embodiment of the invention.

[0041] Figure 2 is a flowchart of a non-real-time data processing method for image or video data on a device using rechargeable battery according to a particular embodiment of the invention, in which data processing method is data compression.

[0042] Figure 3 is a block diagram of a device using rechargeable battery that uses a non- real-time data processing method according to one embodiment of the invention.

DETAILED DESCRIPTION OF THE INVENTION

[0043] The invention is understood more clearly below and described according to various particular embodiments with reference to the accompanying drawings. However, various embodiments are merely possible configurations in which specific features, as described above, is performed individually or omitted. Similar elements illustrated in the drawings are referred to by the same reference number. The description related to similar elements in the drawings may be omitted.

[0044] In actual implementation, according to one preferred embodiment of the invention, a non-real-time data processing method for image or video data on a device using rechargeable battery includes the steps illustrated in Fig. 1.

[0045] First, it should be understood that the device using rechargeable battery 1 is usually mobile device such as mobile phone, smart phone, tablet (e.g., iPad) that has integrated camera and is provided with specialized IC for processing image and video data. Usually, due to limitation in design related to battery size and increased data processing demand, although the devices is provided with specialized IC for processing image, video data, battery life of these devices is generally not suitable to run high efficiency compression algorithms for image and video data in long term, for example, H.265/HEVC (High Efficiency Video Coding) compression standard. Even if the device runs the algorithms to compress video, image with these compression standards, the device will have the battery life reduced and will be heated quickly. The device using rechargeable battery 1 is installed a control software to perform the non-real-time data processing method according to the invention.

[0046] In Fig. 1, in step Sl 1, the device using rechargeable battery 1 starts the non-real time data processing method.

[0047] Next, in step S12, the device using rechargeable battery 1 receives image or video data captured by an integrated camera.

[0048] Next, in step S13, the device (using rechargeable battery) 1 determines whether the device is currently charged by an external source. If it is true, the device using rechargeable battery 1 performs data processing in step S14. If it is false, after a time period customized by the designer or the user, the device 1 returns to step S13.

[0049] Following step S14 is step S15, in which the device 1 determines whether the data processing is completed. If it is true, the device 1 stores the processed data in step S16 and ends the method in step S17. If it is false, the device 1 returns to step S13.

[0050] Fig. 2 illustrates another embodiments of the invention, in which the data processing method is a data compressing method. [0051] The method starts in step S21, the device 1 then receives the image or video data from its integrated camera in step S22.

[0052] Next, in step S23, image or video data is preliminarily compressed to be easily stored in the memory. This data is marked to be re-compressed at least one more time using a compression standard that has higher efficiency.

[0053] Next, in step S24, the device (using rechargeable battery) 1 determines whether the device is currently charged by an external source. If it is true, the device using rechargeable battery 1 re-compresses image or video data using high efficiency compression algorithm in step S241. If it is false, the device 1 temporarily stores the image or video data that was preliminarily compressed in step S242. Until the method is ended, after each time period customized by the designer or the user, the device 1 returns to step S23.

[0054] Following step S24 is step S25, in which the device 1 determines whether the data compression is completed. If it is true, the device 1 stores the compressed data in step S26 and ends the method in step S27. If it is false, the device 1 returns to step S24.

[0055] According to this data processing method, when the user does not use the device 1 and this device is charged by the external source (plugin battery charging), or when the electronic device needs to be charged by the external source, as soon as being charged by the external source (not Portable Power Bank), the step of re-compressing the image or video data will be triggered.

[0056] According to an example embodiment, for image compression, image compression standard such as WebP or BPG may be used, these are advanced efficient image compression method having high image compression ratio. For digital video compression, high efficiency video compression standard such as HEYC may be used. [0057] Preliminary compression of image or video data in real-time is to guarantee that the captured image and video data is not interrupted, and the user can immediately view, query, and use the data.

[0058] In addition, in order to explain more clearly, the non-real-time image or video data processing method according to the invention is different in that, the processor of the device using rechargeable battery is configured to:

[0059] limit the data processing speed, i.e., still maintaining the high computation complexity to achieve high data compression efficiency, but limit the processing speed of the IC of the device using rechargeable battery to guarantee that the temperature of the device does not exceed the safety temperature threshold of this device; and

[0060] the compression process may be temporarily stopped and resumed when continue charging the device using rechargeable battery with the external source.

[0061] Limiting the data processing speed of the IC is to guarantee that the temperature of the device using rechargeable battery in general and the temperature of the battery in particular do not increase too high that may result in explosion risk or cause discomfort to the user.

[0062] At the same time, maintaining high efficiency compression algorithm, for example, compression algorithm according to WebP/BPG standard for image compression, and compression algorithm according to HEVC standard provides optimal effect of saving memory without impacting the battery life.

[0063] The non-real-time image or video data processing method discussed above is merely an example embodiment of the invention. Scope and principle of the invention are not limited to this method. [0064] Other aspects of the invention are further described below (not shown in the drawings).

[0065] According to one particular aspect of this embodiment, the method further includes a step of processing the data only when a screen is off.

[0066] According to one particular aspect of this embodiment, the method further includes a step of processing the data only when a user does not input data into the device during a particular time period. The data input method may be typing on a key, touching sensors or buttons. Example time period can be any value from 1 minute to 1 hour.

[0067] According to one particular aspect of this embodiment, the method further includes a step of limiting the data processing speed such that the data processing operation does not impact other operations of the device, e.g., synchronizing data from the device to the Internet.

[0068] According to one particular aspect of this embodiment, the method further includes a step of processing the data only when the data processing does not occupy Random Access Memory (RAM) of the running applications.

[0069] According to one particular aspect of this embodiment, the method further includes a step of not processing the data when the device is charged by Portable Power Banks.

[0070] According to one particular aspect of this embodiment, the method further includes a step of limiting the data processing speed such that the energy required for processing is sufficiently provided by the external source, therefore the processing operation does not reduce battery life.

[0071] According to one particular aspect of this embodiment, the method further includes a step of limiting the data processing speed such that the battery life is always above a value predefined by the device, this value may be redefined by the user. For example, this value may be from 90% to 100%.

[0072] In addition to applying the non-real-time data processing method according to the invention to compress data, the invention can also be applied to analyze, identify, or other form of data processing.

[0073] According to another aspect of this embodiment, the data processing method is an image recognition method to extract useful information from the data, wherein data processing is recognizing image to extract useful information from the data, or classifying the data to enable a user to mine the data more efficiently.

[0074] According to another aspect of this embodiment, the data processing method is a data analyzing method, wherein analyzing data is to suggest the user to make a data processing decision such as backup or delete redundant data.

[0075] Additionally, according to another embodiment, Fig. 3 illustrates a block diagram of a device using rechargeable battery 1 to perform the non-real-time data processing method according to another embodiment of the invention.

[0076] As depicted in Fig. 3, the device using rechargeable battery 1 includes at least the following components: a rechargeable battery 11, an image processing IC or chip 12, a screen 13, a microcontroller unit (MCU) 14, a camera 15, a microphone 16, a speaker 17, a temperature sensor 18, an input component 19, other sensors 110, and memory 111. The MCU 14 of the device using rechargeable battery 1 is configured to control the components of the device using rechargeable battery 1 to perform the non-real -time data processing method of the invention according to one of the embodiments and aspects discussed above. Examples of the device using rechargeable batery 1 may be mobile phone, smart phone, tablet, even laptop using rechargeable batery, etc.

[0077] Additionally, the data processing method according to the invention may be implemented in the form of a computer program stored in a computer program product, a computer readable medium, the computer program includes instructions that cause suitable hardware components to perform the non-real-time data processing method according to the invention.

[0078] In general, the present technology according to the invention relates to using external power source to process complicated computation operations, as well as limiting the processing speed of IC/chip to not make the temperature of the device using rechargeable batery exceed the safe temperature.

[0079] It should be understood that different embodiments of the invention can be installed or implemented using computing devices (processors). Example computing devices or processors can be multi-functional processor, Digital Signal Processor (DSP), Application Specific Integrated Circuit (ASIC), Field Programmable Gate Array (FPGA), or other programmable logic devices, etc. Various embodiments of the invention can also be implemented or illustrated by a combination of these devices.

[0080] In addition, different embodiments of the inventions can also be implemented by software modules, these modules are executed by processors or executed directly in hardware. In addition, it is also possible to combine software modules and hardware. The software modules can be stored in any form of computer-readable storage media, e.g., RAM, EPROM, EEPROM, flash memory, register, hard disk, CD-ROM, DVD, etc. [0081] It should be noted that, the specific features of different embodiments may belong to a specific element or may be optionally integrated in other elements of the invention.

[0082] Person skilled in the art should understand that there are various alternatives and modifications that can be made with the implementations of the invention without departing the principle and scope of the invention. Therefore, these embodiments are considered in all aspects and not to limit the invention.

TECHNICAL EFFECTS OF THE INVENTION

[0083] The method set forth herein allows the device using rechargeable battery to apply data processing method that has highest efficiency while still guarantees that the battery life is comparable to other existing processing methods. For example, compressing image or video using the best technologies allows for maximally reducing the data size while completely avoid reducing the battery life as compared to existing methods. In addition to compression, the method is also applicable to analyzing, identifying, or other form of data processing.