FIELD OF THE INVENTION

The present invention relates to de-identifying privacy sensitive data on dynamic composition screens. Particularly, the present invention relates to a device and a method for de-identification of information to be displayed.

BACKGROUND OF THE INVENTION

There is a trend that for interventions more and more advanced equipment is used from a multitude of producers. Typically, each of the advanced equipment systems will have its own display output where privacy related data can be shown. From a clinical user perspective, there is a strong need to have these displays controlled in a single integrated fashion.

A wide range of personal data is shown on these displays relating to patients, referring physicians, the hospital staff or other parties involved in the treatment of the patient or involved in room support.

The information is made available for various legitimate business purposes, including patient safety, clinical support for the treatment at hand, billing, or the efficiency and effectiveness of the clinical workflow in the hospital, e.g. contrast allergies, accumulated X-ray dose, room scheduling information, or information for establishing the patients' identify, this to avoid patient mix-ups.

US 2012/0266255 Al describes a method, a computer program product and a system for masking sensitive data and, more particularly, to dynamically de-identifying sensitive data from a data source for a target application, including enabling a user to selectively alter an initial de-identification protocol for the sensitive data elements via an interface.

US 2004/0193901 Al describes a system and a method for the dynamic configuration of patient tags and masking types for de-identifying patient data during image export from a picture archiving and communication system diagnostic workstation

US 2005/0268094 Al describes systems and processes for assembling de- identified patient healthcare data records in a longitudinal database. The systems and processes may be implemented over multiple data suppliers and common database facilities while ensuring patient privacy. At the data supplier locations, patient-identifying attributes in the data records are placed in standard format and then doubly encrypted using a pair of encryption keys before transmission to a common database facility.

US 2010/074525 Al describes that, in an image containing sensitive data, OCR may be applied to recognize textual characters in the image. The recognized characters may be de-identified for example by means of masking. It is further described to identify associations between different strings, for example rows of a table; masking may be applied based upon an association between strings.

SUMMARY OF THE INVENTION

There may be a need to improve devices used for displaying information.

These needs are met by the subject-matter of the independent claims. Further exemplary embodiments are evident from the dependent claims and the following description.

An aspect of the present invention relates to a device for de-identification of information to be displayed, the device comprising a tracking module, which is configured to track a change in screen composition of the information to be displayed; a preset module, which is configured to define privacy presets for different information sources, each preset containing a reference screen size and at least one reference area for de-identification; and a de-identification module configured to, .

A further, second aspect of the present invention relates to a medical imaging system comprising a device according to the first aspect of the present invention or according to any embodiment of the present invention.

A further aspect of the present invention relates to a method for de- identification of information to be displayed, the method comprising the steps of tracking a change in screen composition of the information to be displayed by means of a tracking module; defining privacy presets for different information sources, each preset containing a reference screen size and at least one reference area for de-identification and applying, based on a change of the screen composition, at least one de-identification element to the

information to be displayed, in accordance with the defined privacy presets corresponding to information sources being active for the screen composition.

The present invention advantageously provides a continuous tracking of changes in screen composition of the information to be displayed. Within the scope of the invention, a "screen composition" indicates a composition or layout of a screen displaying information from one or more active information sources; for example, the screen

composition indicates a screen position and size of video sources and/or streams being displayed, as well as an identification of the active sources. A change in screen composition could thus include repositioning, resizing and/or re-arranging views of one or more information sources on the screen and/or a change in active information sources.

Further, the present invention advantageously allows that privacy presets are provided for information sources; preferably, a preset may be provided for each information source, from which the information may need to be displayed on the screen. Each preset contains a reference screen size and at least one reference de-identification area to protect. Preferably, a size and position of a reference de-identification area are provided in relation to the reference screen size.

The information about which sources are active, as well as the actual size and the position of the active sources as they are laid out within the screen composition, is used to apply one or more de-identification elements to the information to be displayed on the screen.

In particular, for an active source, one or more de-identification elements can be added to the information to be displayed in accordance with the privacy preset

corresponding to that source. For this purpose, preferably the information on current size and position of the source within the screen composition is combined with the reference screen size and reference de-identification area of the privacy present for that source. Thus, size and position for the areas to which de-identification elements are to be applied can be calculated, such that these elements effectively render any sensitive information on the screen

unrecognizable.

The invention may be applied in connection with a video composition

(sub)system, which generates a composite output image based on input images from different video information sources. Such composite output image may then be displayed on one or more displays. Generating a composite output image typically involves scaling the different input sources and positioning them in different viewports in the output image. In this case, advantageously, the invention allows for suitable de-identification elements to be applied correctly within the composite image, based on the privacy profiles for each of the active sources being displayed in the composite image.

As a de-identification element, a semi-transparent or non-transparent area may be overlaid on top of the calculated area to cover the personal information to be displayed. Alternatively, the de-identification element may include distorted image information of the calculated area itself, the distortion having been effected by means of a locally applied deforming, warping or blurring algorithm.

The present invention advantageously provides a detection of changes in screen composition or layout to be done by explicit system interface, which is used to determine one or more active video sources within the screen composition, as well as a size and a position of the determined active video sources. Further, the present invention advantageously uses dynamic recognition of unique elements in the video sources and further provides means to locate their size or position.

The present invention advantageously allows that the targeted optical character recognition of the sensitive information is performed using information about the current patient.

The present invention may advantageously provide a device which is configured to share video data from remote parties, where an explicit interface is used. A video compositor subsystem may be used to compose a video output containing video images of multiple video input sources, one or more of which may be third party video input sources. An example of such a video compositor may be any video subsystem.

The remote access subsystem uses a copy of the composited video output and makes the copy available. Data may be stored locally as movie data or image snapshot data.

According to an embodiment of the present invention, the tracking module is configured to track a layout change or a screen composition change.

According to an embodiment of the present invention, the preset module is configured to define privacy presets for different information to be displayed pertaining to distinct individuals. Thus, when different system users have different authorization levels, the privacy presets may include, for one or more reference areas for de-identification, an indication for which ones of the different authorization levels corresponding de-identification elements are to be applied. Thus, for example, patient data in a source may be visible to a phyisician, but a de-identification element may be applied rendering said patient data unrecognizable when information from that source is used for example for training purposes.

According to an exemplary embodiment of the present invention, the de- identification module is configured to apply a de-identification element by altering a transparency of a display area to be de-identified.

According to an exemplary embodiment of the present invention, the tracking module comprises a system interface, which is configured to determine one or more active video sources within the screen composition and , to determine a size and a position of the one or more active video sources.

According to an exemplary embodiment of the present invention, the tracking module is configured to recognize alphanumeric characters. For example, an optical character recognition (OCR) algorithm may be applied to the recognized characters to identify text strings potentially relating to sensitive information.

According to an exemplary embodiment of the present invention, the tracking module may be configured to perform face recognition.

Thus, specific privacy-sensitive image information that may not have been included in the privacy preset for an input source can still be recognized. Preferably, information regarding the recognized privacy-sensitive information is provided by the tracking module to the de-identification module, so that suitable de-identification elements may be applied to display areas corresponding to the recognized information, in addition to any de-identification element being applied based on the privacy profile.

Accordingly, the tracking module may also be used to recognize and de-identify specific privacy- sensitive information in case no privacy profile is available for a certain input source. Alternatively, when no privacy profile is available for an active input source, a de-identification element may be applied that effectively renders the entire view for that information source unregonizable. Such a de-identification element could also include a warning to a user that a privacy profile needs to be defined or obtained for that particular input source. According to an exemplary embodiment of the present invention, the tracking module is configured to track the information to be displayed on a video composer system.

According to an exemplary embodiment of the present invention, the tracking module is configured to track the information to be displayed on a user-interface-element.

A computer program performing the method of the present invention may be stored on a computer-readable medium. A computer-readable medium may be a floppy disk, a hard disk, a CD, a DVD, an USB (Universal Serial Bus) storage device, a RAM (Random Access Memory), a ROM (Read Only Memory) and an EPROM (Erasable Programmable Read Only Memory). A computer-readable medium may also be a data communication network, for example the Internet, which allows downloading a program code.

The methods, systems and devices described herein may be implemented as software in a Digital Signal Processor, DSP, in a micro-controller or in any other side- processor or as hardware circuit within an application specific integrated circuit, ASIC, CPLD or FPGA. The present invention can be implemented in digital electronic circuitry, or in computer hardware, firmware, software, or in combinations thereof, e.g. in available hardware of medical imaging devices or in new hardware dedicated for processing the methods described herein.

A more complete appreciation of the invention and the attendant advantages thereof will be more clearly understood by reference to the following schematic drawings, which are not to scale, wherein:

Figure 1 shows a schematic diagram of dynamic composition screens displaying healthcare information for explaining the present invention;

Figure 2 shows a schematic diagram of dynamic composition screens displaying healthcare information for explaining the present invention;

Figure 3 shows a schematic diagram of screen content comprising personal healthcare information for explaining the present invention;

Figure 4 shows a schematic diagram of screen content comprising personal healthcare information for explaining the present invention;

Figure 5 shows a schematic diagram of screen content comprising personal healthcare information for explaining the present invention;

Figure 6 shows a schematic diagram of a video compositor system according to an exemplary embodiment of the present invention;

Figure 7 shows a schematic diagram of an illustration of display items for a video source for explaining the present invention;

Figure 8 shows a schematic diagram of an illustration of display items for a video source for explaining the present invention;

Figure 9 shows a schematic diagram of a device for de-identification of information to be displayed according to an exemplary embodiment of the invention;

Figure 10 shows a schematic diagram of a device for de-identification of information to be displayed according to an exemplary embodiment of the invention; and

Figure 11 shows a schematic flow-chart diagram of a method for de- identification of information to be displayed according to an exemplary embodiment of the invention.

DETAILED DESCRIPTION OF EMBODIMENTS

The illustration in the drawings is purely schematical and does not intend to provide scaling relations or size information. In different drawings, similar or identical elements are provided with the same reference numerals. Generally, identical parts, units, entities or steps are provided with the same reference symbols in the description.

Figure 1 shows a schematic diagram of dynamic composition screens displaying healthcare information for explaining the present invention. Figure 1 shows dynamic composition screens which display relevant personal data. The dynamic

composition screens may comprise a medical imaging system 200 comprising a device 1 for de-identification of information to be displayed. The personal data may allow associating a person's identity from being connected with displayed information.

Figure 2 shows a schematic diagram of dynamic composition screens displaying healthcare information for explaining the present invention. The dynamic composition screens may comprise a medical imaging system 200 comprising a device 1 for de-identification of information to be displayed. Figure 2 shows dynamic composition screens which are accessible by multiple people.

Figure 3 shows a screenshot, screen capture or screen-cap as comprising the visible items displayed. In Figure 3, personal information PI, e.g. a patient name: John Doe, or the birth day of the patient, 02.04.1951 representing information to be de-identified may be shown in the screen. Different personal patient details may be displayed on the screen. According to an embodiment of the present invention, a continuous tracking of changes in screen composition is performed.

Privacy presets are looked up for the recognized information sources. Each preset contains a reference screen size and the reference de-identification area to protect. The information about the actual size and the position of the sources may be used to scale and/or position the reference de-identification area. Subsequently de-identification elements such as a semi-transparent or deformed or non-transparent area may be overlaid on top of the calculated areas within the screen composition, as shown in Figure 4.

Figure 4 shows a schematic diagram of screen content comprising personal healthcare information for explaining the present invention. Figure 4 shows de-identified screens numbered 1, 2, 3 and 4. In particular, the displayed screen composition comprises a 'main' viewport numbered 1, and three smaller viewports at the right side of the composed screen numbered 2, 3 and 4. In viewports 1 and 3, de-identification elements, in particular non-transparent areas, were overlaid on top of sensitive information. In Figure 4, the personal information PI as present in Figure 3, in this example the name and date of birth of the patient, was de-identified, that is, de-identification elements were applied that mask the sensitive information from the display.. According to an exemplary embodiment of the present invention, an actual scaling of the video sources within the viewport may be performed. Based on the reference areas and sizes in the privacy profiles corresponding to each active information source, a corresponding adaptation of the de-identified personal information PI in the changed screen composition may be calculated and applied

Figure 5 shows a schematic diagram of screen content comprising personal healthcare information for explaining the present invention. Figure 5 shows pre-de-identified screens comprising data which shall be protected using a de-identification routine. The data shown in Figure 5 may comprise personal information PI, e.g. a patient name: John Doe, representing information to be de-identified. Thus, in accordance with the invention, for the information source being shown in Figure 5, a privacy profile is defined or obtained which comprises information on the size and position of the two areas containing personal information PI in relation to a reference size and position for the information display for this source.

Figure 6 shows a schematic diagram of a video compositor system according to an exemplary embodiment of the present invention. Figure 6 depicts a typical system setup to share the video data with remote parties, where an explicit interface is used. The video compositor subsystem VCS used to compose a composition video output CVO containing the video images of multiple (3rd) party video input sources VIS. An example of such a video compositor subsystem is shown in Figure 6. A copy of the composition video output CCVO is sent to a remote access subsystem RAS, which is accessible over a remote access link RAL. The remote access subsystem RAS may be coupled to the video compositor subsystem VCS via a data interface DI.

According to an exemplary embodiment of the present invention, a remote access subsystem RAS may use a copy of the composited video output and makes this available remotely.

According to an exemplary embodiment of the present invention, a video compositor subsystem VCS may have a data interface DI that can be used by the remote access subsystem RAS to retrieve the location, sizes of the different video sources on the composited output image. The video compositor subsystem VCS is provided with a device for de-identification of information to be displayed, in accordance with the invention. Thus, it can be ensured no sensitive data is distributed outside the system through data interface DI. Alternatively, the remote access subsystem RAS is provided with a de-identification device, in which case original video data is provided through the data interface DI and subsequently de-identified in the remote access subsystem RAS.

Figure 7 shows a schematic diagram of an illustration of display items for a video source.

Each video source of a plurality of video sources has a distinct user interface element, also known as features or unique features.

Unique features UF may be the name of an application, distinct icon, font, color, etc. These unique features UF may be put into a reference set and then feature recognition is performed on the composite screen of a video source VS using this reference set. When a feature may be recognized, its scale and its location on the composite screen describe the current location and the size of the video source. Multiple features of the video source VS can be put into the reference set in order to increase the accuracy of its position identification. This information may then be used to build privacy profiles in accordance with the invention.

Preferably, a targeted optical character recognition may be conducted:

typically sensitive information can be easily extracted via standard interfaces from the lab or departmental systems, e.g.: Digital Imaging and Communications in Medicine systems, abbreviated DICOM systems, via HL7 data exchange, Health Level-7 or HL7 refers to a set of international standards for transfer of clinical and administrative data between Hospital information systems.

When the values of case sensitive information are known, optical character recognition can be applied to the composite screen to find its exact location and/or size on the screen, e.g.: Patient Name: John Doe, Date of birth: 01-02-1943, Social security number: 1234456, or further information. The optical character recognition does not require any knowledge about video sources, and the entire composite screen is treated as one source.

Figure 8 shows a schematic diagram of an illustration of display items for a video source for explaining the present invention.

According to an exemplary embodiment of the present invention, a comparison with privacy preset is conducted. Privacy presets may be looked up for the video sources upon each change in the system layout. These privacy presets may be predefined for the most widely used sources or can be added by users for new sources. The preset should contain the reference size of the video source and the related sizes and positions of one or multiple areas for applying de-identification elements, called reference area RA, see Figure 8. According to an exemplary embodiment of the present invention, de- identification elements may have complex shapes. The shape can be often expressed as the set of rectangles or triangles.

According to an exemplary embodiment of the present invention, any unknown source can be entirely covered by a de-identification element, providing a warning to a user that no privacy preset is available and/or inviting user to define a new preset.

According to an exemplary embodiment of the present invention, a de- identification overlay is generated. The actual size of the video sources are matched with the reference size and used to scale the related areas to which de-identification is to be applied. The areas are finally overlaid with suitable de-identification elements such as semi- transparent (deformed) or not transparent shape as described in the above. The semi- transparent deformed overlay provides the advantage that changes on the screen are visible while preserving the privacy.

The present invention advantageously provides that the de-identification of information will also help to prevent privacy breaches in cases where the information is lost, stolen or accessed by unauthorized third parties. For example, if a USB key containing de- identified information is lost, it is unlikely that the person who finds the information would have the motive or capacity to attempt to re-identify the individuals in the data set— it is more likely that there would be no invasion of privacy.

Figure 9 shows a schematic diagram of a device for de-identification of information to be displayed according to an exemplary embodiment of the invention. The device 1 for de-identification of information to be displayed may comprise a tracking module 10, a preset module 20, and a de-identification module 30.

The tracking module 10 may be configured to track a change of a screen composition including the information to be displayed.

The preset module 20 may be configured to define privacy presets for different information sources, each preset containing a reference screen size and at least one reference area for de-identification.

The de-identification module 30 may be configured to, based on a change of the screen composition, apply at least one de-identification element to the information to be displayed, in accordance with the defined privacy presets corresponding to information sources being active for the screen composition .

Figure 10 shows a medical imaging system 200 comprising a device 1 for de- identification of information to be displayed. Figure 11 shows a schematic flow-chart diagram of a method for de- identification of information to be displayed.

The method may comprise the following steps:

As a first step of the method, a step S 1 of tracking a change of a screen composition including the information to be displayed. The tracking step may be conducted by means of a tracking module 10 of a de-identification device according to the invention.

As a second step of the method, a step S2 of defining privacy presets for different information sources, each preset containing a reference screen size and at least one reference area for de-identification. The defining step may be conducted by means of a preset module 20 of a de-identification device according to the invention.

As a third step of the method, a step S3 of applying, based on a change of the screen composition, at least one de-identification element to the information to be displayed, in accordance with the defined privacy presets corresponding to information sources being active for the screen composition.

It has to be noted that embodiments of the invention are described with reference to different subject-matters. In particular, some embodiments are described with reference to method type claims whereas other embodiments are described with reference to the device type claims.

However, a person skilled in the art will gather from the above and the following description that, unless otherwise notified, in addition to any combination of features belonging to one type of subject-matter also any combination between features relating to different subject-matters is considered to be disclosed with this application.

However, all features can be combined providing synergetic effects that are more than the simple summation of the features.

While the invention has been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered illustrative or exemplary and not restrictive; the invention is not limited to the disclosed embodiments. Other variations to the disclosed embodiments can be understood and effected by those skilled in the art and practicing the claimed invention, from a study of the drawings, the disclosure, and the appended claims.

In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. A single processor or controller or other unit may fulfill the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage. Any reference signs in the claims should not be construed as limiting the scope.