MACHINE LEARNING

BACKGROUND

[0001] The disclosure relates generally to machine learning (ML), and more particularly, to user privacy protection using a computational model.

[0002] Different types of personal electronic devices (e.g., smartphone, laptop, personal computer (PC), tablet, etc.) may be used to obtain and communicate various types of multimedia files. These multimedia files may include, e.g., image files, video files, and text files, just to name a few. In some instances, multimedia files may include sensitive content, e.g., such as personal, proprietary, and/or confidential information. Due to the ever-increasing number of multimedia files that can be maintained and communicated by personal electronic devices, there is an increased chance of inadvertently communicating a multimedia file that contains sensitive content to another user.

SUMMARY

[0003] The disclosure relates generally to machine learning (ML), and more particularly, to user privacy protection using a computational model.

[0004] According to one embodiment of the present disclosure, an apparatus for user privacy protection on a user device is disclosed. The apparatus may include a memory and at least one processor coupled to the memory and configured to perform various operations associated with user privacy protection. For example, the at least one processor may be configured to obtain at least one multimedia file. The at least one processor may be further configured to input the at least one multimedia file into a computational model generated using a machine learning (ML) dataset containing a plurality of sensitive content variations, the computational model configured to identify sensitive content. The at least one processor may be also configured to determine whether the at least one multimedia file potentially contains sensitive content based at least in part on an output from the computational model. The at least one processor may be also configured to request first confirmation when it is determined that the at least one multimedia file potentially contains sensitive content. The at least one processor may be also configured to maintain the at least one multimedia file in an encrypted zone when the first confirmation is received.

[0005] According to another aspect of the present disclosure, a method of user privacy protection on a user device. The method may include obtaining at least one multimedia file. The method may further include inputting the at least one multimedia file into a computational model generated using a ML dataset containing a plurality of sensitive content variations, the computational model configured to identify sensitive content. The method may also include determining whether the at least one multimedia file potentially contains sensitive content based at least in part on an output from the computational model. The method may further include requesting first confirmation when it is determined that the at least one multimedia file potentially contains sensitive content. The method may also include maintaining the at least one multimedia file in an encrypted zone when the first confirmation is received.

[0006] According to yet another aspect of the present disclosure, a non-transitory computer-readable medium storing computer executable code for user privacy protection on a user device is disclosed. The non-transitory computer-readable medium may include code to obtain at least one multimedia file. The non-transitory computer-readable medium may also include code to input the at least one multimedia file into a computational model generated using a ML dataset containing a plurality of sensitive content variations, the computational model configured to identify sensitive content. The non-transitory computer-readable medium may also include code to determine whether the at least one multimedia file potentially contains sensitive content based at least in part on an output from the computational model. The non-transitory computer-readable medium may also include code to request first confirmation when it is determined that the at least one multimedia file potentially contains sensitive content. The non-transitory computer-readable medium may also include code to maintain the at least one multimedia file in an encrypted zone when the first confirmation is received.

BRIEF DESCRIPTION OF THE DRAWINGS

[0007] The accompanying drawings, which are incorporated herein and form part of the specification, illustrate the presented disclosure and, together with the description, further serve to explain the principles of the disclosure and enable a person of skill in the relevant art(s) to make and use the disclosure.

[0008] FIG. 1 illustrates a block diagram illustrating an example of a computer system of user privacy protection on a user device, according to embodiments of the disclosure.

[0009] FIG. 2 illustrates a block diagram illustrating a computational model training system, according to embodiments of the disclosure.

[0010] FIG. 3 illustrates a depiction of an example of a training sample used by the computational model training system of FIG. 2, according to embodiments of the disclosure. [0011] FIG. 4 illustrates a block diagram of an exemplary system for user privacy protection using a computational model, according to embodiments of the disclosure.

[0012] FIGs. 5A and 5B illustrate a flow chart of a first exemplary method of user privacy protection, according to embodiments of the disclosure.

[0013] FIG. 6 illustrates a flow chart of a second exemplary method of user privacy protection, according to embodiments of the disclosure.

[0014] Embodiments of the present disclosure will be described with reference to the accompanying drawings.

DETAILED DESCRIPTION

[0015] Although specific configurations and arrangements are discussed, it should be understood that this is done for illustrative purposes only. A person skilled in the pertinent art will recognize that other configurations and arrangements can be used without departing from the spirit and scope of the present disclosure. It will be apparent to a person skilled in the pertinent art that the present disclosure can also be employed in a variety of other applications.

[0016] It is noted that references in the specification to “one embodiment,” “an embodiment,” “an example embodiment,” “some embodiments,” “certain embodiments,” etc., indicate that the embodiment described may include a particular feature, structure, or characteristic, but every embodiment may not necessarily include the particular feature, structure, or characteristic. Moreover, such phrases do not necessarily refer to the same embodiment. Further, when a particular feature, structure, or characteristic is described in connection with an embodiment, it would be within the knowledge of a person skilled in the pertinent art to effect such feature, structure, or characteristic in connection with other embodiments whether or not explicitly described.

[0017] In general, terminology may be understood at least in part from usage in context.

For example, the term “one or more” as used herein, depending at least in part upon context, may be used to describe any feature, structure, or characteristic in a singular sense or may be used to describe combinations of features, structures or characteristics in a plural sense. Similarly, terms, such as “a,” “an,” or “the,” again, may be understood to convey a singular usage or to convey a plural usage, depending at least in part upon context. In addition, the term “based on” may be understood as not necessarily intended to convey an exclusive set of factors and may, instead, allow for existence of additional factors not necessarily expressly described, again, depending at least in part on context.

[0018] As will be disclosed in detail below, the user privacy protection systems and methods described herein employ pre-trained ML models to automatically detect sensitive content in multimedia files. Examples of a multimedia file that contains sensitive content may include, e.g., personal images, videos, audio files, and/or text that include nudity/sex, violence, gore, the image of a person’s driver license or passport, company trade secrets, confidential/proprietary information. Multimedia files identified with sensitive content may be moved to an encrypted zone rather than being maintained with other multimedia files that do not contain sensitive content. For privacy protection, the ML recognized sensitive contents will be encrypted, and they will not be automatically uploaded to the cloud for back-up without the user’s express permission. The present systems and methods may also prompt the user with a warning message before the end- user tries to send or share a multimedia file with sensitive content. Compared with traditional user devices, the systems and methods disclosed herein reduce the chance that a user may inadvertently sent a multimedia file that contains sensitive content to an unintended friend, colleague, family member, or stranger.

[0019] Additional novel features will be set forth in part in the description which follows, and in part will become apparent to those skilled in the art upon examination of the following and the accompanying drawings or may be learned by production or operation of the examples. The novel features of the present disclosure may be realized and attained by practice or use of various aspects of the methodologies, instrumentalities, and combinations set forth in the detailed examples discussed below.

[0020] FIG. 1 is a block diagram illustrating an exemplary user privacy protection system

100 (hereinafter “exemplary system 100”), according to some embodiments of the disclosure. Exemplary system 100 may be implemented by, e.g., apparatus 200 described below in connection with FIG. 2. Exemplary system 100 includes an input pre-processing module 102, an identification module 104, a dispatching module 106, and a plurality of classification-based sensitive content modules 108 (e.g., 108-1, 108-2...108-n, etc.), each of which is configured to perform classification-based sensitive content detection using deep context. In some embodiments, exemplary system 100 may be implemented using a pipeline architecture to combine other sensitive content identification methods, such as image classification and predefined rule -based methods, with the classification-based method to further improve the performance of exemplary system 100. As shown in FIG. 1, exemplary system 100 may further include a sensitive content determination module 110 configured to determine whether an input multimedia file 116 potentially contains sensitive content.

[0021] Input pre-processing module 102 may be configured to receive and process an input multimedia file 116 (hereinafter “multimedia file 116”)· Multimedia file 116 may include any type of media file that includes at least one media element, e.g., such as audio, video, image, and/or text. Multimedia file 116 may be obtained directly, for example, from a user device (e.g., smartphone, laptop, PC, tablet, intemet-of-things (IoT) device, smart device, etc.) that is associated with exemplary system 100. For example, exemplary system 100 may be one component of a user device that also includes, e.g., a camera configured to obtain one or more digital pictures, digital videos, and/or digital audio files. The user device may additionally and/or alternatively include a word processing system configured to generate and/or maintain text-based documents.

[0022] Input pre-processing module 102 may pre-process multimedia file 116 in various manners, e.g., including determining the type of multimedia file (e.g., picture, video, text, audio, etc.). The multimedia file type may be determined based on, e.g., the file extension (e.g., .jpeg, .mp3, .mp4, .png, .gif, .docx, etc.) of multimedia file 116. In some embodiments, multimedia file 116 may be analyzed in the context of subsections (e.g., a portion of an image, frame/segment of a video, sentence or phrase of text, a segment of audio, etc.). Input pre processing module 102 may partition multimedia file 116 into subsections that each may be treated as a separate unit for subsequent processing. For example, an image of a person can be partitioned into portions related to the person’s anatomy (e.g., head/neck/shoulders, arms, torso, leg, etc.) for subsequent processing. In certain implementations, multimedia file 116 may be partitioned into subsections by recognizing the beginning and/or end of different elements or regions of interest. For example, when multimedia file 116 is an image, input pre-processing module 102 may search for demarcations and/or changes in color and/or shape that may be associated with different regions (e.g., anatomy, landscape, vehicles, animals, foreground, background, toys, weapons, etc.). With respect to a video, input pre-processing module 102 may search for changes in scenes (e.g., moving from one room to another, a change of people, movement of people, a change in tone of voice, a change of lighting, etc.). More generally, images and/or videos may be partitioned into halves, thirds, quadrants, etc., rather than by elements and/or regions of interest. With respect to text, input pre-processing module 102 may search for certain punctuations, such as period, semicolon, question mark, or exclamation mark, as the indicators of the end of a sentence. The above examples of partitioning are given for illustrative purposes only. One of the ordinary skills would understand that multimedia file 116 can be partitioned in any way known in the art without departing from the scope of the present disclosure.

[0023] In some embodiments, input pre-processing module 102 may not partition multimedia file 116 into subsections. In this embodiment, subsequent processing may be performed on the entire multimedia file 116 rather than subsections. The following examples will be described with respect to processing the entire multimedia file 116 to determine whether it potentially includes sensitive content, rather than its subsections, for simplicity. However, the same or similar operations may be performed on subsections to determine whether any of the subsections potentially contain sensitive content without depart from the scope of the present disclosure.

[0024] Identification module 104 may be configured to analyze the multimedia file 116 to identify one or more elements therein. An element as used herein may include, e.g., a person, clothing, a body part, an object, a plant, an animal, a vehicle, a building, a landscape, a cityscape, etc. For each element identified in multimedia file 116, identification module 104 may be configured to generate a corresponding element indicator. Multimedia file 116 and its associated indicator(s) may be passed to dispatching module 106.

[0025] Dispatching module 106 may be configured to dispatch multimedia file 116 to one or more classification-based sensitive content module(s) 108 based at least in part on the element indicator(s). For example, for an image of a person, dispatching module 106 may send multimedia file 116 to a classification-based sensitive content module 108 that handles image-based sensitive content associated with people (e.g., nudity, inappropriate gestures, etc.). On the other hand, for an image of a person holding a weapon at a shooting range, dispatching module 106 may be configured to dispatch multimedia file 116 to a classification-based sensitive content module 108 that handles image-based sensitive content associated with people and weapons. Additionally and/or alternatively, for an image of a person holding a weapon at a shooting range, dispatching module 106 may be configured to dispatch multimedia file 116 to a first classification-based sensitive content module 108 that handles image-based sensitive content associated with people and a second weapons classification-based sensitive content module that handles image-based sensitive content associated with weapons.

[0026] In some embodiments, for each type of sensitive content (e.g., violence, profanity, inappropriate gestures, inappropriate clothing, nudity, proprietary information, confidential information, attorney work product, financial information, etc.), a computational model 120-1, 120-2, 102-n may be independently trained and used by corresponding classification-based sensitive content module 108-1, 108-2, 108-n, respectively. Thus, each classification-based sensitive content module 108 may be associated with one specific type of sensitive content and may be configured to determine whether multimedia file 116 potentially contains sensitive content based at least in part on an output from its corresponding computational model 120.

[0027] Classification-based sensitive content module 108 may be configured to estimate a probability that multimedia file 116 includes the sensitive content based on an output of the corresponding computational model 120. Each classification-based sensitive content module 108 that was sent multimedia file 116 may generate a score based on the likelihood of sensitive content. [0028] By way of example and not limitation, assume multimedia file 116 includes a person standing at a shooting range holding a gun and wearing a baseball hat that has an expletive written on the front. Here, identification module 104 may be configured to generate a person tag, weapon tag, and a profanity tag that are sent with multimedia file 116 to dispatch module 106. Furthermore, assuming classification-based sensitive content module 1 108-1 is configured to identify people/weapons in an image, and classification-based sensitive content module 2 108-2 is configured to identify people/written profanity in an image, multimedia file 116 may be dispatched to these two modules 108-1, 108-2 for evaluation. Because computational model 120-1 identifies the weapon pictured in multimedia file 116, classification-based sensitive content module 2 108- 2 may generate a high-value score indicating a high probability of sensitive content, which is then output to sensitive content determination module 110. Moreover, because computational model 120-2 identifies the expletive on the baseball hat in the image, classification-based sensitive content module 2 108-2 may also output a high-value score indicating a high probability of sensitive content, which may be output to sensitive content determination module 110. Sensitive content determination module 110 may sum the scores received from the classification-based sensitive content modules 108, and compare the summation to a threshold.

[0029] When the summation meets the threshold, then sensitive content determination module 110 may trigger an output that requesting confirmation and/or rejection of the sensitive content by a user to determine whether to maintain the multimedia file in an encrypted zone (user response indicates sensitive content) or an unencrypted zone (user response indicates no sensitive content). On the other hand, when the summation does not meet the threshold, then sensitive content determination module 110 may trigger the multimedia file to be maintained in an unencrypted zone of the user device.

[0030] FIG. 2 is a block diagram of an exemplary apparatus 200, according to some embodiments of the disclosure. The exemplary apparatus 200 may include the user privacy protection system 100 described above in connection with FIG. 1. Examples of apparatus 200 include, e.g., a cellular phone, a smart phone, a session initiation protocol (SIP) phone, a mobile station (STA), a laptop, a personal computer (PC), a desktop computer, a personal digital assistant (PDA), a satellite radio, a global positioning system, a multimedia device, a video device, a digital audio player, a camera, a game console, a tablet, a smart device, a wearable device (e.g., smart watch, wireless headphones, etc.), a vehicle, an Intemet-of-Things (IoT) device, or any other similarly functioning device.

[0031] As shown in FIG. 2, the exemplary apparatus 200 may include a processing element, such as processor(s) 202, which may execute program instructions for the exemplary apparatus 200. The exemplary apparatus 200 may also include display circuitry 204, which may perform graphics processing and provide display signals to the display 242. Display circuitry 204 may be configured to generate a confirmation request when sensitive content determination module 110 determines that the overall score from the classification-based sensitive content modules 108 meets the threshold indicating the possibility that multimedia file 116 contains sensitive content. Display 242 may be configured to output the confirmation request so that the user can confirm whether multimedia file 116 contains sensitive content.

[0032] The processor(s) 202 may also be coupled to memory management unit (MMU)

240, which may be configured to receive addresses from the processor(s) 202 and translate the addresses to address locations in memory (e.g., memory 206, Flash memory 210, ROM, etc.) and/or to address locations in other circuits or devices, such as the display circuitry 204, radio 230, connector interface 220, and/or display 242. The MMU 240 may be configured to perform memory protection and page table translation or set up. In some embodiments, the MMU 240 may be included as a portion of the processor(s) 202. In some embodiments, the address locations generated by MMU 240 may indicate an encrypted zone or an unencrypted zone in memory 206, Flash memory 210, etc. Multimedia files maintained in the encrypted zone may be encrypted such that they may be accessed using a password, while multimedia files maintained in the unencrypted zone may be accessed without first entering a password.

[0033] As shown, the processor(s) 202 may be coupled to various other circuits of the exemplary apparatus 200. For example, the exemplary apparatus 200 may include various types of memory, a connector interface 220 (e.g., for coupling to the computer system), the display 242, wireless communication circuitry (e.g., for Wi-Fi, Bluetooth (BT), Bluetooth Low Energy (BLE), cellular, etc.), and/or wired communication circuitry. The exemplary apparatus 200 may include a plurality of antennas 235a, 235b, 235c, 235d, for performing wireless communication with, e.g., wireless devices.

[0034] In certain aspects, the exemplary apparatus 200 may include hardware and software components (a processing element) configured to perform operations associated with the user privacy protection, e.g., using the techniques described in connection with any FIGs. 1 and 3-6. [0035] The exemplary apparatus 200 may be configured to implement part or all of the techniques described herein in connection with any of FIGs. 1 and 3-6, e.g., by executing program instructions stored on a memory medium (e.g., a non-transitory computer-readable memory medium) and/or through hardware or firmware operation. In other embodiments, the techniques described herein in connection with any of FIGs. 1 and 3-6 may be at least partially implemented by a programmable hardware element, such as a field -programmable gate array (FPGA), and/or an application- specific integrated circuit (ASIC).

[0036] In certain aspects, radio 230 may include separate controllers configured to control communications for various respective radio access technology (RAT) protocols. For example, as shown in FIG. 2, radio 230 may include a WLAN controller 250 configured to control wireless local area network (WLAN) communications, a short-range communication controller 252 configured to control short-range communications, and a wireless wide area network (WWAN) controller 256 configured to control WWAN communications. In certain aspects, the exemplary apparatus 200 may store and execute a WLAN software driver for controlling WLAN operations performed by the WLAN controller 250, a short-range communication software driver for controlling short-range communication operations performed by the short-range communication controller 252, and/or a WWAN software driver for controlling WWAN operations performed by the WWAN controller 256. In certain implementations, a first coexistence interface 254 (e.g., a wired interface) may be used for sending information between the WLAN controller 250 and the short-range communication controller 252. In certain other implementations, a second coexistence interface 258 may be used for sending information between the WLAN controller 250 and the WWAN controller 256. In certain other implementations, a third coexistence interface 260 may be used for sending information between the short-range communication controller 252 and the WWAN controller 256. In some aspects, one or more of the WLAN controller 250, the short- range communication controller 252, and/or the WWAN controller 256 may be implemented as hardware, software, firmware, or some combination thereof. In certain configurations, the WLAN controller 250 may be configured to communicate with a second device in a wireless personal area network (WPAN) using a WLAN link using all of the antennas 235a, 235b, 235c, 235d. In certain other configurations, the short-range communication controller 252 may be configured to communicate with at least one second device in a WPAN using one or more of the antennas 235a, 235b, 235c, 235d. In certain other configurations, the WWAN controller 256 may be configured to communicate with a second device in a WPAN using all of the antennas 235a, 235b, 235c, 235d. [0037] FIG. 3 is a block diagram illustrating a computational model training system 300, according to some embodiments. Computation model training system 300 includes a model training module 302 configured to train each computational model 120 for a specific type of sensitive content over a set of training samples 304 based on an objective function 306 using a training algorithm 308. A training sample 304 may include datasets of multimedia files (e.g., images, videos, audio clips, text, etc.) that have been pre-labeled. In some embodiments, each training sample 304 may be a native training sample or a learner training sample. A native training sample as disclosed herein includes a plurality of multimedia files (e.g., image, video, audio, text, etc.) without sensitive content, as opposed to a learner training sample that includes a plurality of multimedia files with sensitive content.

[0038] For example, model training module 302 may begin the training process by loading a pretrained model, e.g., such as a TensorFlow model. The training process comprises a bottleneck phase and a training phase. During the bottleneck phase, training sample 304 is loaded and the pixel values are used as input, or features, for the frozen layers of the pretrained model. The frozen layers include all layers in the neural network up to the penultimate layer, also referred to as “the bottleneck layer.” These layers are referred to as frozen because no training may occur on these layers and operations are pass-through. At these frozen layers, the lower-level patterns that help a model differentiate between the different classes may be computed. The larger the number of layers, the more computationally intensive is this step. Fortunately, since this is a one-time calculation, the results can be cached and used in later runs when experimenting with different parameters.

[0039] Once the output values from the bottleneck phase are computed, they are used as input to retrain the final layer of computational model 120. This process is iterative and runs for the number of times specified by the parameters of computational model 120. During each run, the loss and accuracy are evaluated by model training module 302. Then, the appropriate adjustments are made to improve the associated computational model 120 with the goal of minimizing the loss and maximizing the accuracy. Once training is finished, two computational model formats are output. One of them is the .pb version of the model and the other is the .zip ML.NET serialized version of the model. Although the training process described above is in connection with image classification, the same or similar steps may be applied to video classification, audio classification, or text classification without departing from the scope of the present disclosure.

[0040] FIG. 4 illustrates a block diagram of an exemplary system 400 for user privacy protection, according to embodiments of the disclosure. In some embodiments, as shown in FIG. 4, system 400 may include a communication interface 402, a processor 404, a memory 406, and a storage 408. In some embodiments, system 400 may have different modules in a single device, such as an integrated circuit (IC) chip (e.g., implemented as an application-specific integrated circuit (ASIC) or a field-programmable gate array (FPGA)), or separate devices with dedicated functions. In some embodiments, one or more components of system 400 may be located in a cloud or may be alternatively in a single location (such as inside a mobile device) or distributed locations. Components of system 400 may be in an integrated device or distributed at different locations but communicate with each other through a network (not shown). Consistent with the present disclosure, system 400 may be configured to identify potentially sensitive content in multimedia files.

[0041] Communication interface 402 may send data to and receive data from databases via communication cables, a Wireless Local Area Network (WLAN), a Wide Area Network (WAN), wireless networks such as radio waves, a cellular network, and/or a local or short-range wireless network (e.g., BT), or other communication methods. In some embodiments, communication interface 402 may include an integrated service digital network (ISDN) card, cable modem, satellite modem, or a modem to provide a data communication connection. As another example, communication interface 402 may include a local area network (LAN) card to provide a data communication connection to a compatible LAN. Wireless links can also be implemented by communication interface 402. In such an implementation, communication interface 402 can send and receive electrical, electromagnetic or optical signals that carry digital data streams representing various types of information.

[0042] Consistent with some embodiments, communication interface 402 may receive a computational model 401 from a database or while in the factory. Computational model 401 may include a plurality of computational models each trained to identify a particular type of sensitive content, e.g., as described above in connection with FIGs. 1 and 3. Communication interface may also receive at least one multimedia file 411a. Multimedia file 411a may be obtained by a digital camera associated with system 400, generated using a word processor associated with system 400, or from an external device (e.g., via text message, email, short-range communication, cellular data communication, etc.), just to name a few. Communication interface 402 may further provide the received data to memory 406 and/or storage 408 for storage (e.g., computational model 401) or to processor 404 for processing (e.g., multimedia file 411a).

[0043] Processor 404 may include any appropriate type of general-purpose or special- purpose microprocessor, digital signal processor, or microcontroller. Processor 404 may be configured as a separate processor module dedicated to identifying sensitive content in multimedia files. Alternatively, processor 404 may be configured as a shared processor module for performing other functions in addition to user privacy protection.

[0044] Memory 406 and storage 408 may include any appropriate type of mass storage provided to store any type of information that processor 404 may need to operate. Memory 406 and storage 408 may be a volatile or non-volatile, magnetic, semiconductor, tape, optical, removable, non-removable, or other types of storage device or tangible (i.e., non-transitory) computer-readable medium including, but not limited to, a read-only memory (ROM), a flash memory, a DRAM, and a SRAM. Memory 406 and/or storage 408 may be configured to store one or more computer programs that may be executed by processor 404 to perform functions disclosed herein. For example, memory 406 and/or storage 408 may be configured to store program(s) that may be executed by processor 404 to perform user privacy protection of a user device (e.g., apparatus 200).

[0045] In some embodiments, memory 406 and/or storage 408 may also store various computational models used to identify sensitive content within multimedia file 41 la. Memory 406 and/or storage 408 may update computational model(s) to include new data points when sensitive content is correctly and/or incorrectly identified in multimedia file 41 la by system 400.

[0046] Moreover, memory 406 and/or storage 408 may include an unencrypted zone and an encrypted zone. The unencrypted zone may be configured to store or maintain multimedia file(s) that do not include sensitive content. On the other hand, the encrypted zone may be configured to store or maintain multimedia file(s) that contain sensitive content. Access to the multimedia file(s) in the encrypted zone may be encrypted and/or password-protected so that only users with access to the password may gain access to the multimedia file(s) in the encrypted zone. [0047] As shown in FIG. 4, processor 404 may include multiple modules, such as a computational model unit 444, a first confirmation unit 446, a second confirmation unit 448, an encryption unit 450, and the like. These modules (and any corresponding sub-modules or sub units) can be hardware units (e.g., portions of an integrated circuit) of processor 404 designed for use with other components or software units implemented by processor 404 through executing at least part of a program. The program may be stored on a computer-readable medium, and when executed by processor 404, it may perform one or more functions. Although FIG. 4 shows units 444-450 all within one processor 404, it is contemplated that these units may be distributed among different processors located closely or remotely with each other.

[0048] In some embodiments, units 442-450 of FIG. 4 may execute computer instructions to identify sensitive content contained in multimedia file 411a.

[0049] FIGs. 5A and 5B illustrate a flowchart of a first exemplary method 500 for user privacy protection of a user device, according to embodiments of the disclosure. Method 500 may be performed by system 400 and particularly processor 404 or a separate processor not shown in FIG. 4. Method 500 may include steps 502-528 as described below. It is to be appreciated that some of the steps may be optional, and some of the steps may be performed simultaneously, or in a different order than shown in FIGs. 5A and 5B. FIGs. 4, 5A, and 5B will be described together. [0050] Referring to FIG. 5A, at 502, communication interface 402 may obtain at least one multimedia file 411a. Multimedia file 41 lamay include an image file, a video file, an audio file, a text file, or a mix thereof. System 400 may be part of an apparatus (e.g., user device) that includes, among others, a digital camera system, a digital video recording system, a digital audio recording system, and a word processing system, just to name a few. In this example, multimedia file 411a may be obtained by one of the digital or word processing systems and then received by system 400 via communication interface 402. Additionally and/or alternatively, multimedia file 41 lamay be obtained from an external device not associated with but in communication with system 400. For example, multimedia file 411a may include an image file sent via text message, short message service (SMS), email, or short-range communication from an external device. In this example, multimedia file 411a may be obtained via wired or wireless communication.

[0051] At 504, communication interface 402 may input multimedia file 411a into a computational model generated using an ML dataset containing a plurality of sensitive content variations. For example, communication interface 402 may input multimedia file 411ainto computational model unit 444, which may include the computational model 401 generated using an ML dataset (e.g., as described above in connection with FIG. 3) and received by communication interface 402.

[0052] At 506, computational model unit 444 may determine whether the at least one multimedia file potentially contains sensitive content based at least in part on an output from computational model 401, e.g., as described above in connection with FIG. 1. When it is determined (at 506) that multimedia file 41 la does not contain sensitive content, the operation may move to 508. Otherwise, when it is determined (at 506) that multimedia file 411a contains sensitive content, the operation may move to 510.

[0053] At 508, memory 406 and/or storage 408 may maintain multimedia file 411a in an unencrypted zone when it is determined (at 506) that multimedia file 411a does not contain sensitive content.

[0054] At 510, first confirmation unit 446 may request first confirmation when it is determined (at 506) that multimedia file 41 lapotentially contains sensitive content. For example, first confirmation unit 446 may generate an output (e.g., pop-up window, audio output, vibration, etc.) that alerts the user to the potential for sensitive content in multimedia file 411a. The user may interact with display 410 to confirm whether multimedia file 411a contains sensitive content. For example, a pop-up window may be displayed on display 410 and may request confirmation whether multimedia file 411a contains sensitive content. A user can interact with the pop-up window (e.g., a ‘yes’ and ‘no’ box) to confirm whether multimedia file 411acontains sensitive content. A signal indicating the user’s response 413a may be sent from communication interface 402 to first confirmation unit 446.

[0055] At 512, first confirmation unit 446 may determine whether multimedia file 411a contains sensitive content based on the user’s response 413a to first confirmation request 405. When the user’s response 413a indicates that multimedia file 411a does not include sensitive content, then the operation may move to 514, where memory 406 and/or storage 408 may maintain multimedia file 41 la in an unencrypted zone. Otherwise, when the user’s response 413a indicates that multimedia file 411a does contain sensitive content, the operation may move to 516, where memory 406 and/or storage 408 may maintain multimedia file 411a in an encrypted zone. Encryption unit 450 may encrypt multimedia file 411a prior to it being moved to the encrypted zone. Additionally and/or alternatively, encryption unit 450 may generate and/or maintain a password that grants access to multimedia file 41 la in the encrypted zone.

[0056] Referring to FIG. 5B, at 518, communication interface 402 may receive a request

407 (hereinafter “send request 407”) to send a multimedia file 411b in the encrypted zone to an external device (not shown). Multimedia file 411b may be the same as multimedia file 411a or may be a different file maintained in the encrypted zone.

[0057] In certain implementations, at 520, multimedia file 411b may be sent to the external device automatically but without the password used to access multimedia file 411b in order to protect a user’s privacy in case of inadvertent transmission.

[0058] However, in certain other implementations, send request 415 may be sent to the encryption unit 450, which may determine whether multimedia file 411b is maintained in the encrypted zone of memory 406 and/or storage 408. Encryption unit 450 may access a lookup table and/or register that indicates which multimedia files (maintained by memory 406 and/or storage 408) are in the encrypted zone and which are in the unencrypted zone to determine the location of multimedia file 411b. When encryption unit 450 determines that multimedia file 411b is in the encrypted zone, a signal indicating as much may be sent by encryption unit 450 to second confirmation unit 448.

[0059] At 522, second confirmation unit 448 may request second confirmation 409, which indicates that multimedia file 411b contains user-confirmed sensitive content, and requests confirmation as to whether multimedia file 41 lb is to be sent to the external device. For example, second confirmation unit 448 may generate an output (e.g., pop-up window, audio output, vibration, etc.) that alerts the user to his or her request to send multimedia file 411b with sensitive content to an external device (not shown). The user may interact with display 410 to confirm whether multimedia file 41 lb is to be sent to the external device. For example, the user can interact with the pop-up window (e.g., a ‘yes’ and ‘no’ box) on display 410 to confirm whether multimedia file 41 lb is to be sent to the external device. A signal indicating the user’s response 413b may be sent from communication interface 402 to second confirmation unit 448.

[0060] At 524, second confirmation unit 448 may determine whether multimedia file 411b is to be sent to the external device bases at least in part on user response 413b. When it is determined (at 524) that multimedia file 41 lb is not to be sent to the external device, the operation may stop, and multimedia file 41 lb is not sent. On the other hand, when it is determined (at 524) that multimedia file 41 lb is to be sent to the external device, the operation may move to 526. [0061] At 526, communication interface 402 may send multimedia file 411b (with or without the encrypted password) to the external device. Multimedia file 411b may be sent via wired or wireless communication.

[0062] At 528, computational model unit 444 may update the computational model with a datapoint associated with multimedia file 411a and an indication as to whether the user confirmed (at 512) the inclusion of sensitive content therein.

[0063] FIG. 6 illustrates a flowchart of a second exemplary method 600 for user privacy protection of a user device, according to embodiments of the disclosure. Method 600 may be performed by system 400 and particularly processor 404 or a separate processor not shown in FIG. 4. Method 600 may include steps 602-614 as described below. It is to be appreciated that some of the steps may be optional, and some of the steps may be performed simultaneously, or in a different order than shown in FIG. 6. FIGs. 4 and 6 will be described together.

[0064] Referring to FIG. 6, at 602, communication interface 402 may input a multimedia file 411a into a computational model 401 (of computational model unit 444), which may be generated using an ML dataset containing a plurality of sensitive content variations, e.g., as described above in connection with FIGs. land 3.

[0065] At 604, computational model unit 444 may determine whether multimedia file 411a potentially contains sensitive content based at least in part on an output from computational model 401, e.g., as described above in connection with FIG. 1. When it is determined (at 604) that multimedia file 411a does not contain sensitive content, the operation may move to 612. Otherwise, when it is determined (at 604) that multimedia file 411a contains sensitive content, the operation may move to 606.

[0066] At 606, display 410 may output a warning (e.g., first confirmation request 405) to the user when it is determined (at 604) that multimedia file 411a potentially contains sensitive content. For example, first confirmation unit 446 may generate a warning (e.g., pop-up window, audio output, vibration, etc.) that alerts the user to the potential for sensitive content in multimedia file 41 la. The user may interact with display 410 to confirm whether multimedia file 41 la contains sensitive content. For example, a pop-up window may be displayed on display 410 and may request the user to confirm whether multimedia file 411a contains sensitive content. A user can interact with the pop-up window (e.g., a ‘yes’ and ‘no’ box) to confirm whether multimedia file 411acontains sensitive content. A signal indicating the user’s response 413a may be sent from communication interface 402 to first confirmation unit 446.

[0067] At 608, At 512, first confirmation unit 446 may determine whether the user accepted the advice (at 606) regarding sensitive content. In other words, first confirmation unit 446 may determine whether multimedia file 411a contains sensitive content based on the user’s response 413a to first confirmation request 405. When the user’s response 413a indicates that multimedia file 411a does not include sensitive content, then the operation may move to 612, where memory 406 and/or storage 408 may maintain multimedia file 41 la in an unencrypted zone. Otherwise, the user’s response 413a indicates that multimedia file 411a does contain sensitive content; the operation may move to 610, where memory 406 and/or storage 408 may maintain multimedia file 41 lain an encrypted zone. Encryption unit 450 may encrypt multimedia file 411a prior to or when it is moved to the encrypted zone. Additionally and/or alternatively, encryption unit 450 may generate and/or maintain a password that grants access to multimedia file 411a in the encrypted zone.

[0068] At 612, memory 406 and/or storage 408 may maintain multimedia file 411a in an unencrypted zone when it is determined (at 604 or 608) that multimedia file 411a does not contain sensitive content.

[0069] At 614, communication interface 602 may upload multimedia file 411b, which in this example may be multimedia file 41 la, to an unencrypted cloud-based service for future access by the user or others (e.g., colleagues, friends, etc.). For example, multimedia file 411b may be uploaded to a shared picture folder or a shared work folder depending on whether multimedia file 41 lb is personal or work-related.

[0070] According to one embodiment of the present disclosure, an apparatus for user privacy protection on a user device is disclosed. The apparatus may include a memory and at least one processor coupled to the memory and configured to perform various operations associated with user privacy protection. For example, the at least one processor may be configured to obtain at least one multimedia file. The at least one processor may be further configured to input the at least one multimedia file into a computational model generated using an ML dataset containing a plurality of sensitive content variations, the computational model configured to identify sensitive content. The at least one processor may be also configured to determine whether the at least one multimedia file potentially contains sensitive content based at least in part on an output from the computational model. The at least one processor may be also configured to request first confirmation when it is determined that the at least one multimedia file potentially contains sensitive content. The at least one processor may be also configured to maintain the at least one multimedia file in an encrypted zone when the first confirmation is received.

[0071] In some embodiments, the at least one processor may be further configured to maintain the at least one multimedia file in an unencrypted zone when the first confirmation is received. The unencrypted zone may be either local or remote to the user device.

[0072] In some embodiments, the at least one processor may be further configured to receive a request to send the at least one multimedia file in the encrypted zone to an external device. [0073] In some embodiments, the at least one processor may be further configured to request second confirmation to send the at least one multimedia file in the encrypted zone to the external device when the request is received. In some embodiments, the at least one processor may be further configured to send the at least one multimedia file in the encrypted zone and an associated password when the second confirmation is received.

[0074] In some embodiments, the at least one processor may be further configured to send the at least one multimedia file in the encrypted zone without an associated password.

[0075] In some embodiments, the at least one processor may be further configured to update the computational model with the at least one multimedia file when it is confirmed that the at least one multimedia file contains sensitive content.

[0076] In some embodiments, the at least one multimedia file includes one or more of an image file, a video file, or a text file.

[0077] According to another aspect of the present disclosure, a method of user privacy protection on a user device. The method may include obtaining at least one multimedia file. The method may further include inputting the at least one multimedia file into a computational model generated using an ML dataset containing a plurality of sensitive content variations, the computational model configured to identify sensitive content. The method may also include determining whether the at least one multimedia file potentially contains sensitive content based at least in part on an output from the computational model. The method may further include requesting first confirmation when it is determined that the at least one multimedia file potentially contains sensitive content. The method may also include maintaining the at least one multimedia file in an encrypted zone when the first confirmation is received.

[0078] In some embodiments, the method may further include maintaining the at least one multimedia file in an unencrypted zone when the first confirmation is received. The unencrypted zone may be either local or remote to the user device.

[0079] In some embodiments, the method may further include receiving a request to send the at least one multimedia file in the encrypted zone to an external device.

[0080] In some embodiments, the method may further include requesting second confirmation to send the at least one multimedia file in the encrypted zone to the external device when the request is received. In some embodiments, the method may further include sending the at least one multimedia file in the encrypted zone and an associated password when the second confirmation is received.

[0081] In some embodiments, the method may further include sending the at least one multimedia file in the encrypted zone without an associated password.

[0082] In some embodiments, the method may further include updating the computational model with the at least one multimedia file when it is confirmed that the at least one multimedia file contains sensitive content.

[0083] In some embodiments, the at least one multimedia file includes one or more of an image file, a video file, or a text file.

[0084] According to yet another aspect of the present disclosure, a non-transitory computer-readable medium storing computer executable code for user privacy protection on a user device is disclosed. The non-transitory computer-readable medium may include code to obtain at least one multimedia file. The non-transitory computer-readable medium may also include code to input the at least one multimedia file into a computational model generated using an ML dataset containing a plurality of sensitive content variations, the computational model configured to identify sensitive content. The non-transitory computer-readable medium may also include code to determine whether the at least one multimedia file potentially contains sensitive content based at least in part on an output from the computational model. The non-transitory computer-readable medium may also include code to request first confirmation when it is determined that the at least one multimedia file potentially contains sensitive content. The non-transitory computer-readable medium may also include code to maintain the at least one multimedia file in an encrypted zone when the first confirmation is received.

[0085] In some embodiments, the non-transitory computer-readable medium may also include code to maintain the at least one multimedia file in an unencrypted zone when the first confirmation is received. The unencrypted zone may be either local or remote to the user device. [0086] In some embodiments, the non-transitory computer-readable medium may also include code to receive a request to send the at least one multimedia file in the encrypted zone to an external device.

[0087] In some embodiments, the non-transitory computer-readable medium may also include code to request second confirmation to send the at least one multimedia file in the encrypted zone to the external device when the request is received. In some embodiments, the non-transitory computer-readable medium may also include code to send the at least one multimedia file in the encrypted zone and an associated password when the second confirmation is received.

[0088] In some embodiments, the non-transitory computer-readable medium may also include code to send the at least one multimedia file in the encrypted zone without an associated password.

[0089] In some embodiments, the non-transitory computer-readable medium may also include code to update the computational model with the at least one multimedia file when it is confirmed that the at least one multimedia file contains sensitive content.

[0090] In some embodiments, the at least one multimedia file includes one or more of an image file, a video file, or a text file.

[0091] While the present disclosure has been described herein with reference to exemplary embodiments for exemplary fields and applications, it should be understood that the present disclosure is not limited thereto. Other embodiments and modifications thereto are possible, and are within the scope and spirit of the present disclosure. For example, and without limiting the generality of this paragraph, embodiments are not limited to the software, hardware, firmware, and/or entities illustrated in the figures and/or described herein. Further, embodiments (whether or not explicitly described herein) have significant utility to fields and applications beyond the examples described herein.

[0092] Embodiments have been described herein with the aid of functional building blocks illustrating the implementation of specified functions and relationships thereof. The boundaries of these functional building blocks have been arbitrarily defined herein for the convenience of the description. Alternate boundaries can be defined as long as the specified functions and relationships (or equivalents thereof) are appropriately performed. Also, alternative embodiments may perform functional blocks, steps, operations, methods, etc. using orderings different than those described herein.

[0093] The breadth and scope of the present disclosure should not be limited by any of the above-described exemplary embodiments, but should be defined only in accordance with the following claims and their equivalents.