Technical Field

[001] The present invention relates to ultrasound simulation, and particularly to ultrasound simulation including a medical tool.

Background

[002] Ultrasounds machines are commonly used in a range of medical procedures for both diagnosis and clinical practice. For diagnosis they are used as a diagnostic imaging modality. In clinical practice, the “gold standard” is to use ultrasound in an increasing number of medical procedures. Some examples include the use of ultrasound sonography either for point of care ultrasound (POCUS) and focused assessment with sonography for trauma (FAST), or for assisting a surgeon for part of an ultrasound guided surgical procedure.

[003] The increasing use of medical procedures involving ultrasound has resulted in a need for ultrasound training to extend beyond traditional training approaches. One existing approach for ultrasound training is the use of a training mannequin. Mannequins cannot be easily adapted to different scenarios, especially simulating pathological cases. Mannequin based systems that use a real ultrasound machine allow for the teaching of ultrasound techniques, but require both a mannequin and an ultrasound machine, compounding the expense of the system. As a result, such training may be expensive and resource intensive. For example, with a mannequin based system it may be difficult to show patient specific data. The training may also be limited by the mannequin and use of a predetermined model. For example, it may be difficult to present patients of different body shape, age, gender, etc. other than by providing a mannequin for each permutation of such characteristics. Using an ultrasound machine may require ultrasound machines for training only purposes or may remove ultrasound machines from being used for diagnostic purposes. The result is that additional cost is either spent on purchasing and maintaining ultrasound machines or ultrasound machine availability decreases for diagnostic purposes while providing limited training benefits. Also, clinicians are increasingly having to travel or attend workshops over many days for access to ultrasound and POCUS training. Clinicians also have to review and retain proficiency over time by repeating such expenses or workshops. Foundational training, upskilling, access to training and deskilling are all considerations of the modern clinician with regards to ultrasound which has profound implications for patient safety and outcomes. [004] An alternative approach to ultrasound training has been to use an ultrasound simulator. In one system, ultrasound images may be recorded from patients. The recorded ultrasound images are used to generate an ultrasound image in a simulator. As an operator moves a physical ultrasound probe, the ultrasound simulator may modify the recorded ultrasound image or images to show the effect of moving the ultrasound probe on a patient. However, such approaches may result in behaviour that does not accurately model the behaviour of an ultrasound when used on a patient as the modification of the recorded ultrasound scan images may lack accuracy and not be a true representation of a real ultrasound image of a patient. Further, such approaches may not be able to produce an accurate representation for an ultrasound guided surgical procedure. Ultrasound simulators are also restrictive and narrow in their “scope” of training and practice. From a technology perspective ultrasound simulators don’t have the advantage of being adaptive or real-time while from a clinical perspective ultrasound simulators are limited and lack breadth and depth of training delivery.

Summary

[005] One embodiment includes a method of simulating an ultrasound scan of tissue in real-time, the method comprising: loading a three dimensional deformable soft tissue model of a tissue region for scanning, the deformable soft tissue model being a volumetric model; determining a three dimensional interaction between the deformable soft tissue model and at least one medical tool; modifying the deformable soft tissue model according to the three dimensional interaction and properties of the deformable soft tissue model, the properties including compressible and incompressible regions of the deformable soft tissue model; and generating a simulated scan of the tissue in real-time according to a position of an ultrasound probe and the modified soft tissue model.

[006] In one embodiment, the deformable soft tissue model includes a plurality of particles and each of the plurality of particles are linked using at least one constraint.

[007] In one embodiment, the deformable soft tissue model comprises a mesh and each of the plurality of particles are located at vertices of the mesh.

[008] In one embodiment, a first particle of the plurality of particles is linked to a second particle of the plurality of particles of the deformable soft tissue model using at least one constraint. [009] In one embodiment, the at least one constraint is selected from the set consisting of a distance constraint, a volume constraint and a contact constraint.

[010] In one embodiment, particles in a compressible region of the deformable soft tissue model are connected to adjacent particles with distance constraints and particle in an incompressible region of the deformable soft tissue model are connected to adjacent particles with volume constraints.

[Oil] In one embodiment, a voxel model overlaps the deformable soft tissue model and is used to derive continuously varying material properties of the deformable soft tissue model.

[012] In one embodiment, the voxel model has a resolution different to the deformable soft tissue model.

[013] In one embodiment, a model of the medical tool includes a plurality of particles and a particle of the plurality of particles is linked to a particle of the plurality of particles of the deformable soft tissue model using at least one constraint.

[014] In one embodiment, modifying the deformable soft tissue model comprises: locating particles of the deformable soft tissue model nearby the medical tool; attaching a plurality of particles of the deformable soft tissue model to a plurality of particles of the medical tool using constraints; and determining a new location for the plurality of particles of the deformable soft tissue model and a new location for the plurality of particles of the medical tool base on movement of the medical tool.

[015] In one embodiment, the at least one constraint is selected from the set consisting of a distance constraint, a volume constraint and a contact constraint.

[016] In one embodiment, the medical tool is selected from a set of medical tools consisting of an ultrasound probe, a hypodermic needle, a cannula, a scalpel, sutures, dressing, and forceps.

[017] In one embodiment, the deformable soft tissue model is formed from a scan selected from the set of a CT scan and a MRI scan.

[018] In one embodiment, the modification to the deformable soft tissue model is performed by a position based dynamics solver. [019] In one embodiment, the position based dynamics solver is a Small Steps approach to an extended position based dynamics (XPBD) solver.

[020] In one embodiment, generating the simulated scan further comprises: forming an image slice using material properties of the medical tool and the properties of the deformable soft tissue model, wherein the material properties of the medical tool produce acoustic anomalies associated with the medical tool.

[021] One embodiment includes a virtual ultrasound system for simulating an ultrasound scan of tissue in real-time, the system comprising: a deformable soft tissue model of a tissue region, the deformable soft tissue model being a volumetric model; a model of at least one medical tool; a position based dynamics solver configured to determine a three dimensional interaction between the deformable soft tissue model and the model of the at least one medical tool in real-time, the position based dynamics solver also being configured to modify the deformable soft tissue model according to the three dimensional interaction and properties of the deformable soft tissue model, wherein the properties include compressible and incompressible regions of the deformable soft tissue model; an acoustic simulator configured to generate a simulated scan of the tissue according to a position of an ultrasound probe and the modified soft tissue model; and an output device for displaying the simulated scan of the tissue.

[022] In one embodiment, the deformable soft tissue model includes a plurality of particles and each of the plurality of particles are linked using at least one constraint.

[023] In one embodiment, the deformable soft tissue model comprises a mesh and each of the plurality of particles are located at a vertices of the mesh.

[024] In one embodiment, a first particle of the plurality of particles is linked to a second particle of the plurality of particles of the deformable soft tissue model using at least one constraint.

[025] In one embodiment, the at least one constraint is selected from the set consisting of a distance constraint, a volume constraint and a contact constraint.

[026] In one embodiment, particles in a compressible region of the deformable soft tissue model are connected to adjacent particles with distance constraints and particle in an incompressible region of the deformable soft tissue model are connected to adjacent particles with volume constraints.

[027] In one embodiment, a voxel model overlaps the deformable soft tissue model and is used to derive continuously varying material properties of the deformable soft tissue model.

[028] In one embodiment, the voxel model has a resolution different to the deformable soft tissue model.

[029] In one embodiment, a model of the medical tool includes a plurality of particles and a particle of the plurality of particles of the model of the medical tool is linked to a particle of the plurality of particles of the deformable soft tissue model using at least one constraint.

[030] In one embodiment, the at least one constraint is selected from the set consisting of a distance constraint, a volume constraint and a contact constraint.

[031] In one embodiment, the medical tool is selected from a set of medical tools consisting of an ultrasound probe, a hypodermic needle, a cannula, a scalpel, sutures, dressing, and forceps.

[032] In one embodiment, the deformable soft tissue model is formed from a scan selected from the set of a CT scan and a MRI scan.

[033] In one embodiment, the position based dynamics solver is an extended position based dynamics (XPBD) solver.

[034] In one embodiment, the position based dynamics solver is a Small Steps approach to an extended position based dynamics (XPBD) solver.

[035] In one embodiment, the acoustic simulator further comprises: an image slice generator configured to form an image slice using material properties of the medical tool and the properties of the deformable soft tissue model, wherein the material properties of the medical tool produce acoustic anomalies associated with the medical tool.

Brief Description of Figures

[036] At least one embodiment of the present invention is described, by way of example only, with reference to the accompanying figures. [037] Figure 1 illustrates a functional block diagram of an example processing system that can be utilised to embody or give effect to a particular embodiment;

[038] Figure 2 illustrates an example network infrastructure that can be utilised to embody or give effect to a particular embodiment;

[039] Figure 3 illustrates a dynamic virtual ultrasound in accordance with one embodiment of the invention;

[040] Figure 4 illustrates a simulation workflow for operation of the dynamic virtual ultrasound of Figure 3;

[041] Figures 5A and B illustrate details of a position based dynamics solver of Figure 4; and

[042] Figures 6A, B and C illustrate insertion of a needle into a soft tissue model according to the dynamic virtual ultrasound of Figure 3;

[043] Figures 7A, 7B, 7C, 7D, 7E and 7F illustrate output images from the dynamic virtual ultrasound of Figure 3;

[044] Figures 8 A, 8B, 8C and 8D illustrate output images from the dynamic virtual ultrasound of Figure 3; and

[045] Figure 9 illustrates a method of generating a scanned image from the dynamic virtual of Figure 3.

Detailed Description

[046] Disclosed is method of simulating an ultrasound scan of tissue. The method may comprise loading a three dimensional soft tissue model of a tissue region for scanning. A three dimensional interaction between the soft tissue model and at least one medical tool may be determined. Next, the soft tissue model may be modified according to the three dimensional interaction and properties of the soft tissue model. The properties may include compressible and incompressible regions of the soft tissue model. A simulated scan of the tissue may be generated according to a position of an ultrasound probe and the modified soft tissue model. [047] A dynamic virtual ultrasound simulates imaging and user interactions required to perform diagnostic ultrasound imaging and ultrasound guided procedures. The dynamic virtual ultrasound includes a soft tissue model, also referred to as a deformable soft tissue model, a volumetric soft tissue model or a soft body dynamics soft tissue model which is a soft tissue model simulated using soft body dynamics. The soft tissue model implements tissue mechanics so that soft tissue may be deformed by external influences, such as gravity, applied pressure, medical tools and pressure inside vessels. The soft tissue model is a model of the soft tissue, bones, joints, organs, etc. of the patient using soft body dynamics.

[048] The dynamic virtual ultrasound also includes various modelled medical tools that may interact with and deform the soft tissue model. Also included is a position based dynamics solver for the soft tissue model and interaction between the modelled medical tools and the soft tissue model. The position based dynamics solver may also be referred to as a soft body dynamics solver. An acoustic simulator may also form part of the dynamic virtual ultrasound. The acoustic simulator may be based on a model of a medical ultrasound machine including machine settings, transducer characteristics, soft tissue response and acoustic anomalies. The acoustic simulation may include anomalies found in real ultrasound imaging such as anomalies related to the use of surgical implements, e.g. comet-tails from needle tips. The dynamic virtual ultrasound also has a user interface that may receive user inputs and provide feedback in both visual and haptic forms to the user. As part of the user interface the internal parts of the soft tissue model of a patient may be visualised through the use of a virtual ultrasound probe, as would be used for diagnostic sonography or in ultrasound guided surgical procedures.

[049] A particular embodiment of the present invention can be realised using a processing system, an example of which is shown in Fig. 1. In particular, the processing system 100 generally includes at least one processor 102, or processing unit or plurality of processors, memory 104, at least one input device 106 and at least one output device 108, coupled together via a bus or group of buses 110. In certain embodiments, input device 106 and output device 108 could be the same device. An interface 112 can also be provided for coupling the processing system 100 to one or more peripheral devices, for example interface 112 could be a PCI card or PC card. At least one storage device 114 which houses at least one database 116 can also be provided. The memory 104 can be any form of memory device, for example, volatile or non-volatile memory, solid state storage devices, magnetic devices, etc. The processor 102 could include more than one distinct processing device, for example to handle different functions within the processing system 100.

[050] Input device 106 receives input data 118 and can include, for example, a keyboard, a pointer device such as a pen-like device or a mouse, audio receiving device for voice controlled activation such as a microphone, data receiver or antenna such as a modem or wireless data adaptor, data acquisition card, etc. Input data 118 could come from different sources, for example keyboard instructions in conjunction with data received via a network. Output device 108 produces or generates output data 120 and can include, for example, a display device or monitor in which case output data 120 is visual, a printer in which case output data 120 is printed, a port for example a USB port, a peripheral component adaptor, a data transmitter or antenna such as a modem or wireless network adaptor, etc. Output data 120 could be distinct and derived from different output devices, for example a visual display on a monitor in conjunction with data transmitted to a network. A user could view data output, or an interpretation of the data output, on, for example, a monitor or using a printer. The storage device 114 can be any form of data or information storage means, for example, volatile or non-volatile memory, solid state storage devices, magnetic devices, etc.

[051] In use, the processing system 100 is adapted to allow data or information to be stored in and/or retrieved from, via wired or wireless communication means, the at least one database 116. The interface 112 may allow wired and/or wireless communication between the processing unit 102 and peripheral components that may serve a specialised purpose. The processor 102 receives instructions as input data 118 via input device 106 and can display processed results or other output to a user by utilising output device 108. More than one input device 106 and/or output device 108 can be provided. It should be appreciated that the processing system 100 may be any form of terminal, server, specialised hardware, or the like.

[052] The processing system 100 may be a part of a networked communications system 200, as shown in Fig. 2. Processing system 100 could connect to network 202, for example the Internet or a WAN. Input data 118 and output data 120 could be communicated to other devices via network 202. Other terminals, for example, thin client 204, further processing systems 206 and 208, notebook computer 210, mainframe computer 212, PDA 214, pen- based computer or tablet 216, server 218, etc., can be connected to network 202. A large variety of other types of terminals or configurations could be utilised. The transfer of information and/or data over network 202 can be achieved using wired communications means 220 or wireless communications means 222. Server 218 can facilitate the transfer of data between network 202 and one or more databases 224. Server 218 and one or more databases 224 provide an example of an information source.

[053] Other networks may communicate with network 202. For example, telecommunications network 230 could facilitate the transfer of data between network 202 and mobile, cellular telephone or smartphone 232 or a PDA-type device 234, by utilising wireless communication means 236 and receiving/transmitting station 238. Satellite communications network 240 could communicate with satellite signal receiver 242 which receives data signals from satellite 244 which in turn is in remote communication with satellite signal transmitter 246. Terminals, for example further processing system 248, notebook computer 250 or satellite telephone 252, can thereby communicate with network 202. A local network 260, which for example may be a private network, LAN, etc., may also be connected to network 202. For example, network 202 could be connected with ethemet 262 which connects terminals 264, server 266 which controls the transfer of data to and/or from database 268, and printer 270. Various other types of networks could be utilised.

[054] The processing system 100 is adapted to communicate with other terminals, for example further processing systems 206, 208, by sending and receiving data, 118, 120, to and from the network 202, thereby facilitating possible communication with other components of the networked communications system 200.

[055] Thus, for example, the networks 202, 230, 240 may form part of, or be connected to, the Internet, in which case, the terminals 206, 212, 218, for example, may be web servers, Internet terminals or the like. The networks 202, 230, 240, 260 may be or form part of other communication networks, such as LAN, WAN, ethernet, token ring, FDDI ring, star, etc., networks, or mobile telephone networks, such as GSM, CDMA, 4G, 5G etc., networks, and may be wholly or partially wired, including for example optical fibre, or wireless networks, depending on a particular implementation.

[056] Figure 3 shows modules that may form part of a dynamic virtual ultrasound 300. The modules of the dynamic virtual ultrasound 300 may be executed on a computer such as the processing system 100 communicating over the network 202. Alternatively, the dynamic virtual ultrasound 300 may be implemented on a dedicated hardware system customised for operation as a dynamic virtual ultrasound. [057] A first module of the dynamic virtual ultrasound 300 is a soft tissue model 310 where the tissue of a patient is modelled. The dynamic virtual ultrasound provides a soft tissue model of a patient, or a specific anatomical region of the patient, with which a user of the dynamic virtual ultrasound may interact using medical tools. The soft tissue model may be a volumetric model and in some instances may be a regular 3D voxel model. One advantage of using a volumetric model is that the model aligns with common data formats produced by a CT (Computerized Tomography) scanner or a MRI (Magnetic Resonance Imaging) scanner which may use DICOM (Digital Imaging and Communications in Medicine). The use of a common data format may allow data produced by CT and MRI scanners to form a basis for a soft tissue model with minimal pre-processing, allowing patient- specific data to be used.

[058] The soft tissue model may be built upon a lattice or mesh such as a hexahedral or tetrahedral mesh. The mesh may be used to control deformation of the soft tissue model. The mesh is formed from cells, for example, each hexahedron or tetrahedron, with vertices of the mesh being a location of a particle. When using a position based dynamics solver, such as XPBD, soft bodies may be represented as particles located at vertices of the mesh of the soft tissue model, with the particles connected, or linked, together by constraints such as a contact constraint, distance constraint, volume constraint, position constraint, or other types of constraints. An example of such an arrangement may be seen in Figures 6 A, 6B and 6C where tissue particles, such as the tissue particle 610, are connected together by constraints such as constraint 615. The soft tissue model may be built as follows.

[059] First, a soft tissue model resolution is determined. The soft tissue model and the voxel model may have the same or different resolutions. For example, the voxel model may have voxels that are 0.5mm in size, but a soft tissue model may have resolution of 2mm as the spacing between particles.

[060] In a second step, compressible parts of the soft tissue model, e.g., blood vessels, are connected with distance constraints to adjacent particles to form cells.

[061] In a third step, incompressible parts of the model, e.g. soft tissue, organs, or bone, are connected with volume constraints to form cells. The use of volume constraints allow for conservation of volume for the incompressible parts of the soft tissue model. [062] In a fourth step, particles located in free space regions, e.g. air, of the soft tissue model are connected with very weak distance constraints so that the free space region has little influence on the soft tissue model. However, the presence of the free space regions of the soft tissue model allows the soft tissue model to be restored reliably after being compressed.

[063] For the compressible and incompressible parts of the soft tissue model, a type and stiffness of material inside each of the cells of the mesh may be used, where stiffness refers to a tensile stiffness mechanical property of a material, which is quantified by Young's Modulus. The material stiffness values are used to constrain the solution from the position based dynamics solver. The type and material properties for a cell of the soft tissue model may be determined using a voxel model. That is, the voxels model is used to derive continuously varying material properties for the soft tissue model. The material property of the cell of the soft tissue model is found from the voxel model. For example, a density of the soft tissue model at a particular point in space can be determined by mapping coordinates in the soft tissue model to coordinates in the voxel model. The density of the soft tissue model can then be found using the density from the voxel model.

[064] Material properties at a given voxel of the voxel model are defined by a transfer function:

M _ijk = F(i,j, k) (1)

Where: (i, j, k) identifies a voxel location in the voxel model and Mi _jk is material properties at the voxel. The material properties may include, but is not limited to, stiffness, compressibility and density. A transfer function represents a continuously varying function over a region of space and may be used to provide material properties at specific locations in space for use in the soft tissue model.

[065] Defining the transfer function may be done based on data from a CT or MRI scanner. Some examples of how to determine the transfer function include the following. Firstly, material properties at all locations are known a priori and supplied as a separate voxel dataset. The transfer function simply looks up the material properties in the supplied voxel dataset. One alternative is to manually build the transfer function, mapping values from a CT and/or MRI scan to known material properties. In this example, all regions of bone will have the same material properties that may be found using a look up table. [066] A third alternative may be used for data provided from a CT scan. CT scan data has a relationship between measured intensity and material stiffness. A transfer function may be determined using data points that are of a known substance, such as bone, fat, muscle, and skin, and assign the material stiffness to the substance based on the CT scan intensity. Using the intensity value allows a transfer function to be created from CT scan data and linear interpolation may be used to generate stiffness values located between the intensity values, thus generating approximate stiffness values for each voxel.

[067] Different transfer functions may be used within a soft tissue model/voxel model through the use of segmentation masks. Segmented spatial regions may use different transfer functions. In one example, segmentation masks may be used for regions where CT and/or MRI scan data values overlap for segments with different material properties.

[068] A dynamic solver 320 is implemented as part of the dynamic virtual ultrasound 300 and makes modifications to the soft tissue model 310 based on interactions between the soft tissue model 310 and medical tools such as the ultrasound probe. The dynamic solver 320 uses a position based dynamic solver such as an extended position based dynamics (XPBD) or a variant, such as the Small Steps approach, to modify the soft tissue model 310. A position based dynamics solver may be fast to compute while being reasonably stable. Object positions are updated directly and the dynamic position based dynamics solver 320 calculates resulting velocities after constraining particles based on the distance or volume constraints. In some versions of the dynamic virtual ultrasound the dynamic position based dynamics solver 320 may also calculate acceleration and forces.

[069] The modules of the dynamic virtual ultrasound 300 may communicate using a communications hub 330. The communications hub 330 is connected to each of the modules and may receive and transmit data to each of the modules. Data received by the communications hub 330 may be transmitted to an individual module or broadcast to each of the modules, with the module filtering out unrequired data. In an alternative arrangement of the dynamic virtual ultrasound 300, each module may be directly connected to other modules where transmission of data is required.

[070] An acoustic simulator 340 forms part of the dynamic virtual ultrasound and may be used to generate the ultrasound image that is displayed to a user of the system. The acoustic simulator 340 uses ray tracing techniques to simulate propagation of ultrasound waves in the soft tissue model from the ultrasound probe. When using an ultrasound machine on a patient, artefacts are produced in the resulting imagery due refraction, reflection and absorption of the acoustic energy as the ultrasound interacts with soft tissue. Such artefacts may enhance realism when incorporated into the output of the dynamic virtual ultrasound. The acoustic simulator 340 of the dynamic virtual ultrasound 300 uses ray tracing techniques and applies ray tracing to a soft tissue model to generate the output image.

[071] Although ultrasound propagates in three dimensions, performing ray tracing in 3D may be computationally expensive. The dynamic virtual ultrasound 300 may use an alternative approach to 3D ray tracing to reduce computational requirements and help maintain real-time performance of the dynamic virtual ultrasound 300. Instead of performing 3D ray tracing, ray tracing may be performed only in a plane defined by a position and orientation of the ultrasound probe. Limiting the ray tracing calculation to a plane may greatly reduce computational requirements at the expense of not including out-of-plane artefacts. However, such artefacts may not be significant in practice.

[072] The output image produced by the dynamic virtual ultrasound 300 and displayed to the user of the dynamic virtual ultrasound 300 may be obtained by determining the image from slicing the soft tissue model with the plane defined by the position and orientation of the ultrasound probe. Pixel values for the output image may be obtained by the transfer function described above in relation to equation 1 of the soft tissue model 310. Specifically, the acoustic simulator 340 uses acoustic impedance from the soft tissue model at all locations in the output image to calculate refraction. As relative acoustic impedance may be used to calculate refraction, the relative acoustic impedance may be substituted for a quantity proportional to acoustic impedance. Acoustic impedance is proportional to density, which is one of the material properties of the transfer function of equation 1. The density of soft tissue is also related to intensity in a CT image, which allows the use of a CT image as the source of suitable proxy for acoustic impedance.

[073] Medical tools, as described below in relation to tool models 350, may also be simulated as part of the dynamic virtual ultrasound 300. When a medical tool, such as a hypodermic needle, intersects the plane defined by the position and orientation of the ultrasound probe, the tool may be incorporated into the soft tissue model on which ray tracing is performed to generate the output image.

[074] The dynamic virtual ultrasound 300 also includes the tool models 350. When using the dynamic virtual ultrasound a user may interact with tissue with a number of medical tools. The medical tools are dynamic objects modelled using particles and constraints in the position based dynamics solver. The constraints may be used in two ways. First, a tool may have internal constraints to model the tool. Secondly, interactions between a tool and the soft tissue model may be modelled with additional constraints between a particle of the tool and a particle of the soft tissue model.

[075] Medical tools are dynamic objects that take part in the physics simulation of the dynamic virtual ultrasound. As a result, complex physical interactions between a tool and the tissue may be modelled. An example of a complex physical interaction is the way that a physician may retract and reposition a needle during a regional block procedure; the needle may not be moved arbitrarily as the needle position is constrained by the soft tissue.

[076] Position constraints between a medical tool and soft tissue are handled using the position based dynamics solver in the dynamic virtual ultrasound 300. With suitable constraints created between particles in the medical tool and the soft tissue model, a tool may be positioned based on user input before the position based dynamics solver re -positions the tool. For the example of a physician retracting and repositioning a needle during a regional block procedure, the physician may attempt to move a simulated needle, using a virtual reality controller, in a way that is physically impossible, resulting in the simulated needle being “pushed back” by the position based dynamics solver, according to the soft tissue model, to a realistic location.

[077] Haptic feedback may be generated by the dynamic virtual ultrasound 300 to signal a difference between user movement of a medical tool and an allowed position of the medical tool calculated by the position based dynamics solver. The haptic feedback may be used to align the medical tool, as controlled by the user, with the allowed position of the medical tool. One alternative to using haptic feedback to achieve co-location of the user’s hands and the medical tools is to allow divergence between the location of the hands and the medical tool, but connect the two locations with virtual springs. In one example the virtual springs are modelled by distance constraints to pull the medical tool model in the direction of the hand’s location. The result, once applied by the position based dynamics solver, is a force on the medical tool model in the direction of the hand’s location.

[078] Interaction between a medical tool and the soft tissue model may vary depending on a medical tool type. Examples of medical tool types include an ultrasound probe, a hypodermic needle, a cannula, a scalpel, sutures, dressing, and forceps. [079] An ultrasound probe may be represented in the soft tissue model as a rigid body with embedded particles that control a position of the ultrasound probe. Moving the ultrasound probe causes collisions between particles of the ultrasound probe and particles of the soft tissue model, although collisions with air particles cells may be ignored. The collisions result in the addition of contact constraints between the colliding particles of the soft tissue model and the particles of the ultrasound probe. The contact constraints allow the user to depress the soft tissue of the soft tissue model by pushing the probe. The depression simulates flattening of the skin of a patient under the probe to allow transducers of the ultrasound probe to make contact with the skin of the patient for effective operation of the ultrasound probe.

[080] A user of the dynamic virtual ultrasound may also apply pressure, by pressing down on the ultrasound probe, to aid blood vessel identification. Under pressure from the probe, a vein will compress while an artery will not. The different behaviour between veins and arteries may be seen on an ultrasound display and is an important technique used in many ultrasound guided procedures. The use of volume constraints for incompressible tissue and distance constraints for compressible parts of the soft tissue model allows pressure applied at the skin to transfer to blood vessels and the blood vessels are able to collapse due to the blood vessels material properties. For a soft tissue model with suitable material properties of the different sections, a vein will collapse with less pressure applied than required to compress an artery.

[081] Hypodermic needles and cannulas may be operated by a user of the dynamic virtual ultrasound. The simulation of needles and cannulas allows for regional block procedures to be simulated. As a needle is advanced through soft tissue, the needle drags the soft tissue due to friction between the needle shaft and the soft tissue. The friction between a needle, or cannula, and soft tissue is modelled by the position based dynamics solver and the soft tissue model.

[082] A needle shaft may be represented in the soft tissue model as a rigid body with particles evenly spaced along the length of the shaft. As the needle advances through the soft tissue, distance constraints are added between particles on the needle shaft and adjacent particles of the soft tissue model. The distance constraints have a rest length equal to a distance between the particles at the beginning of a simulation time step. The needle is inserted further by the user, increasing the distance between constrained particles of the needle and soft tissue model. Next, the position based dynamics solver moves particles in the soft tissue model in order to satisfy the distance constraints. As described below, the constraints are discarded at the end of a simulation time step and new constraints are generated during a subsequent simulation time step. This approach allows needle retraction to be handled in the same manner as needle insertion as needle retraction is symmetrical to needle insertion. As a result, no special handling of needle retraction is necessary.

[083] The position based dynamics solver takes into account the volume and distance constraints on the soft tissue model as well as the distance constraints between particles on the needle and the soft tissue model. Linking the needle particles and the soft tissue model particles allows for deformation of the soft tissue model by the needle while maintaining the physical properties encoded in the soft tissue model.

[084] The use of distance constraints between the needle and soft tissue model particles provides a reasonable approximation of kinetic friction, which is a constant friction force when moving at a constant velocity. When a needle is moved at a constant velocity, in the dynamic virtual ultrasound, a constraint function of the distance constraints will return an approximately constant value because a difference between a constraint length before and after the needle is advanced will be approximately the same for all of the needle related constraints. A model of how kinetic friction between soft tissue and a needle shaft varies with velocity may be incorporated into the soft tissue model as all the relevant material properties are known. However, the dynamic virtual ultrasound may use a linear relationship between velocity and friction.

[085] Figures 5A and 5B describe how needle interaction with a soft tissue model may be implemented within the simulation workflow 400 of Figure 4. Figures 5A and 5B will be described below in more detail.

[086] Any constraints added to handle the insertion of a needle or cannula may be discarded at the end of a simulation time step of the position based dynamics solver. As a result, the constraints are determined and added for any subsequent time step. As such, there is no need to remove, update or otherwise manage the lifetime of these constraints.

[087] When needle velocity becomes zero, after deforming the soft tissue model, the soft tissue constraints will try to push the soft tissue back into an original undeformed state. The constraints between the needle and the nearby particles of the soft tissue model are still created for each time step, but the needle will not be advanced as the needle has zero velocity. However, the constraints between the needle and the soft tissue model will oppose the motion of the soft tissue, effectively applying friction to the movement of the soft tissue as the soft tissue returns to an undeformed state.

[088] An example of needle insertion into a soft tissue model will now be described in relation to figures 6 A, 6B and 6C. The diagrams show a 2D deformation of a soft tissue model, although in practice the needle will be inserted into a 3D soft tissue model.

[089] Although not shown in figures 6A, 6B and 6C, the position based dynamics solver will not only move soft tissue particles, but will also move particles of the needle as the soft tissue will “push back” on the needle. That is, the soft tissue will apply a force to the needle particles through the constraints between the needle particles and the soft tissue model particles. When the needle is advanced, the force applied to the needle from the soft tissue is negligible. However, if the needle is moved laterally or pivoted (i.e. swung) by the user then there will be considerable push back from the soft tissue. The push back from the soft tissue model may result in a desired position of the needle, as set by the user, and an actual position of the simulated needle, determined by the position based dynamics solver, being different. As described above, the difference in position between the desired position and the actual position of the simulated needle may be communicated to the user using a feedback mechanism, such as haptic feedback.

[090] Cutting of the soft tissue model by scalpels may be achieved by duplicating a particle that was shared between two cells so that each cell has an independent particle. This allows the two cells to separate where the particle was duplicated. If cutting is to be performed at locations between particles, then cells may be split to allow particles to be added at the appropriate locations. The use of a scalpel may also remove constraints between the duplicated or added particles where the scalpel has passed through the soft tissue model.

[091] The dynamic virtual ultrasound 300 also has a user interface 360 for interactions with a user. The user interface 360 has an input and output aspect that may be implemented in different ways. A requirement for the user interface 360 is that the user is able to move medical tools, such as an ultrasound probe or a needle, in relation to the tissue and to see results of the ultrasound simulation. One way to implement the user interface 360 is to use a virtual reality interface where the user has a virtual reality view via goggles or a display screen. In a virtual reality system the user may provide input through the use of gloves, worn by the user, or handheld controllers, to provide position information for one or two hands of the user. An alternative approach is for the user to pick up an object of the tool, such as a proxy or dummy tool. A position and orientation of the object may be tracked by cameras that form part of the dynamic virtual ultrasound 300 and the position of the object incorporated into the virtual environment. An alternative to a virtual reality approach is to use augmented reality where virtual reality components may be added to a scene. In an augmented reality approach the patient, tools and scan output may be overlaid on a physical environment.

[092] The user interface 360 may include physics-based interactions, where possible, to assist in providing a realistic simulation. Object positions, such as a medical tool position, may be updated directly and the position based dynamics solver calculates resulting velocities, and force or acceleration if required, after constraining object positions based on the model constraints.

[093] A simulation workflow 400 for the dynamic virtual ultrasound will now be described in relation to Figure 4. The simulation workflow 400 may be practiced on a computer such as the processing system 100. The simulation workflow 400 starts with receiving a soft tissue model step 410. As described above, CT or MRI scans may be used to generate the soft tissue model. Generation of the soft tissue model may be performed outside of the dynamic virtual ultrasound or, in some implementations, the soft tissue model may be generated by the dynamic virtual ultrasound.

[094] Next, at a receive user input step 420, the simulation workflow 400 may receive input from a user of the dynamic virtual ultrasound. The user input for the dynamic virtual ultrasound is described above in relation to the user interface 360 of the dynamic virtual ultrasound 300. A new position of an ultrasound probe, and/or other medical tools, is determined by the user interface 360 at a determine new tool location step 430. The new position is determined from an output of the receive user input step 420.

[095] At an apply position based dynamics solver step 440 a position based dynamics solver, such as XPBD, may be applied to the soft tissue model, including the tool position information from the determine new tool location step 430. The apply position based dynamics solver step 440 may operate as described above in relation to the dynamic position based dynamics solver 320.

[096] Execution of the simulation workflow 400 may be parallelised by using two or more threads when executing on the processing system 100. In one example, a first of the threads interacts with the user and proceeds to an update location step 450 where the location of the tool may be updated. As described above, the location of the tool may be updated after the position based dynamics solver determines a position for the tool. The determined position may be different to the position set by the user. One example in which this situation may occur is when the soft tissue model pushes back on a medical tool to reposition the tool. The medical tool position set by the user in the receive user input step 420 is different due to the soft tissue model pushing on the tool. The user interface of the simulation workflow 400 may indicate the updated position of the tool to the user, as described above in relation to the user interface 360. For example, the updated position of the medical tool may be signalled to the user of the dynamic virtual ultrasound using haptic feedback in an apply feedback step 460.

[097] A second of the threads may execute a regularise image slice step 470 and may be executed by the processing system 100. At the regularise image slice step 470 a slice is sampled of the soft tissue model from the receive soft tissue model step 410 and the resulting sliced regularised. The regularise image slice step 470 is performed because ray-tracing a deformed soft tissue model may be computationally expensive compared to ray-tracing a regular soft tissue model. Image slicing is performed along a plane defined by the ultrasound probe on the deformed soft tissue model. The regularised slice is an image of a cross section of the soft tissue model. An image, i.e. a 2D image of pixels, of the slice is formed and material properties of the deformed soft tissue model at each pixel of the image is sampled. The result is a regular 2D image containing the material properties across the slice. The regularised slice is in a form that allows efficient ray tracing. A mechanism for sampling the soft tissue model at a given pixel location, which corresponds to a point in the 3D model space, uses baricentric coordinates of a sample point in a deformed cell of the soft tissue model containing the sample point. The sample point is mapped back to the undeformed soft tissue model. Once mapped to the undeformed soft tissue model the sample point can be mapped to the (i, j, k) space of the input data where the transfer function of equation 1 may be used to evaluate the material properties for the sample location.

[098] A number of acoustic anomalies may occur during ultrasound scanning when a medical tool is inserted into a patient. Such acoustic anomalies may include acoustic shadowing, specular reflections, beam width and side lobe artefacts, needle bevel reflection, and reverberations that include comet tails. An example of an acoustic anomaly created by a medical tool are reverberations created by needles and guidewires. Specular reflections off needles and guidewires when they are near another strong reflector, such as the boundary of a blood vessel, can cause reverberations because acoustic energy reflects back and forth between the tool and the strong reflector before being transmitted back to the ultrasound probe. In the case of a large gauge needle, reverberation can happen within the hollow needle shaft, giving rise to comet-tail artefact that helps identify the needle in short-axis techniques and causes familiar reverberation stemming from the needle shaft in long-axis techniques. Examples of medical tools that are inserted into the soft tissue include hypodermic and echogenic needles, cannulas, guidewires and catheters.

[099] Acoustic anomalies may be reproduced in the dynamic virtual ultrasound as part of the simulation workflow 400. Material properties of any medical tools intersecting the image slice may be incorporated into the 2D image formed in the regularise image slice step 470. As described above, the transfer function of equation 1 may be used to evaluate the material properties for the sample location for each pixel of the image. Incorporating medical tools is done by checking each pixel in the image to determine if a location of a current pixel is inside any part of a medical tool. If the current pixel is determined to be inside a medical tool, the material properties of the tool are used for the pixel, instead of the material properties of the soft tissue model. The material properties of the medical tool may be determined using a transfer function based on the tool, similar to the transfer function of equation 1. Such an approach results in the medical tools being incorporated into the acoustic simulation and produces the acoustic anomalies. The acoustic simulation does not need to treat medical tools explicitly. Because the material properties of the tools are captured in the 2D image that is ray traced, the medical tools, and any anomalies they create, are produced as part of the acoustic simulation.

[0100] At an ultrasound simulation step 480 the imaging slice from the regularise image slice step 470 and the updated tool locations from the update location step 450 are combined before the ultrasound simulation is performed. The updated location of the medical tool, or tools, from the update location step 450 is used as the tool location, instead of the proposed tool location from the determine new tool location step 430, as the updated location of the medical tool takes into account forces applied to the medical tool by the soft tissue model. The ultrasound simulation is performed as described above in relation to the acoustic simulator 340.

[0101] Once the ultrasound simulation step 480 has generated the ultrasound image, the image is displayed to the user of the dynamic virtual ultrasound at an output to user step 490. The ultrasound image, as well as information relating the positions of any medical tools, are displayed according to the selected user interface. As described above, in relation to the user interface 360, the ultrasound image and the medical tool information may be displayed using a virtual reality headset. Alternatively, the medical tool may be physical tool that has position information provided to a user using haptic feedback and the ultrasound image is displayed using a conventional monitor.

[0102] The simulation workflow 400 may return to the receive user input step 420 and loop to receive updated information from the medical tools, update the position based dynamics solver and display updated information to the user.

[0103] A position based dynamics solver 500, that may be used at the apply position based dynamics solver step 440 of the simulation workflow 400, will now be described in relation to Figure 5A. The position based dynamics solver 500 is described for use with an ultrasound probe and a tool, such as a needle, and may be practiced on a computer such as the processing system 100 communicating over the network 202.

[0104] A first step of the position based dynamics solver 500 is a predict position step 510, where positions of particles are predicted based on velocities calculated from a previous simulation time step and external forces, such as gravity. At a collision handling step 520 additional constraints are added between particles of the tool and particles of the soft tissue model. The additional constraints are used to modify the soft tissue model based on movement of the particles of the tool as the tool interacts with the soft tissue. An example of additional constraints used with a needle and a soft tissue model will be described below in relation to Figure 6 A, 6B and 6C.

[0105] Once collisions between the tool and the soft tissue model has been dealt with in the collision handling step 520, updates are made to the soft tissue model at an iterative solve step 530 where the position based dynamics solver processes the particles and constraints of the soft tissue model as well as the particles and constraints between the tool and the soft tissue model. Once the iterative solve step 530 has converged to a solution, the position of the particles in the soft tissue model and the tool are updated at an update step 540. Velocity may also be determined as displacement between the old and new positions of particles is determined with a known time period between the old and new positions. [0106] Further details of the collision handling step 520 specific to needle interaction will now be described with reference to Figure 5B. The collision handling step 520 may be practiced on a computer such as the processing system 100 communicating over the network 202 and be performed in three steps. A first step is a find nearby particles step 560 where particles near to the needle particles are located. The location of nearby particles may be based on locating a nearest particle in the soft tissue model. An alternative is to locate all particles of the soft tissue model located within a predetermined distance. In addition to distance, a direction of movement of the needle may also be taken into consideration for the selection of particles of the soft tissue model. In one example, particles of the soft tissue model close to the needle and in front of the needle particle, based on the direction of movement of the needle, may be selected. One way to select the particle is to form a plane through the needle particle, with the needle being a normal to the plane. The particles in the soft tissue model located on or behind the plane, based on the direction of movement of the needle, may be selected. The result is that a particle of the needle is attached to a particle of the soft tissue model that the needle particle has moved past. If there is no particle on the needle in a suitable location, then a particle of the soft tissue model may be connected to a particle located at a tip of the needle. Selecting a particle of the soft tissue model that the needle particle has passed and linking with constraints may effectively model friction forces between the needle and the soft tissue.

[0107] An attach particles step 570 may then be performed between a particle of the needle and the nearby particles selected in the find nearby particles step 560. The needle particles and the soft tissue model particle may be attached with distance constraints. Finally, at a move needle particles step 580, the needle is moved to a new location. As described above, the iterative solve step 530 will determine new locations for the soft tissue model particles as well as new locations for the needle particles based on the movement of the needle.

[0108] When the position based dynamics solver is a XPBD solver the addition of constraints may be performed during normal XPBD collision handling. The additional constraints are discarded at the end of each simulation. The additional constraints are required to be re- added for each simulation time step.

[0109] An example of a needle being inserted into a soft tissue model will now be described in relation to Figures 6A, 6B and 6C. The three figures show the progress of a simulated needle as the needle is inserted into a soft tissue model over a single simulation time step with Figure 6A and Figure 6B showing progress of the needle into the soft tissue model and Figure 6C showing deformation of the soft tissue model caused by the needle. The deformation of the soft tissue model is determined at the end of the simulation time step. The deformation of the soft tissue model and the position of the needle may be shown on an output display to a user when an ultrasound scan of the soft tissue model and needle is generated.

[0110] A soft tissue model 600, as used in the dynamic virtual ultrasound, will now be described in relation to Figure 6. The soft tissue model 600 has tissue particles, such as tissue particle 610, linked by one or more constraints such as constraint 615. While the soft tissue model 600 is shown as a two dimensional representation, for the sake of simplicity, the actual model used in the dynamic virtual ultrasound is three dimensional.

[0111] A simulated needle 620 is shown as being partially inserted into the soft tissue model 600. That is, the needle 620 has passed through the soft tissue particles and the constraints between the soft tissue particles. The needle 620 has needle particles 625, 670 and 675. The needle 620 is at rest and the soft tissue model 600 is in an undeformed state. A user of the dynamic virtual ultrasound has advanced the needle 620 and the movement will be reflected in deformation of the soft tissue model 600 and the simulated needle 620 position.

[0112] As part of the position based dynamics solver collision handling, nearby particles of the soft tissue model 600, such as tissue particles 632, 634, 642, 644 and 652, are determined. The nearby particles are located either adjacent to, or within a predetermined distance of, the needle 620. For each of the nearby particles, a distance constraint such as tissue needle distance constraints 633, 635, 643, 645 and 653 are formed between the nearby particles and a nearest particle on the needle 620. The constraints are attached between a soft tissue model particle and a needle particle located ahead, with respect to a direction of movement of the needle 620, of the soft tissue particle. If there are no particles on the needle 620 positioned ahead of the soft tissue particle then the soft tissue particle may be attached to a tip of the needle 620.

[0113] Figure 6B shows an update to the soft tissue model 600 used in the dynamic virtual ultrasound where the needle 620 has been pushed further into the soft tissue model 600 to a proposed needle location. That is, the needle 620 has advanced further into the soft tissue model 600 when compared to the needle 620 of Figure 6A. The movement of the needle 620 between Figure 6A and 6B increases a length of the tissue needle distance constraints 633, 635, 643, 645 and 653 added in Figure 6A. During a next iteration of the position based dynamics solver the tissue needle constraints 633, 635, 643, 645 and 653 will pull on the tissue particles 632, 634, 642, 644 and 652 to further deform the soft tissue model 600.

[0114] Figure 6C shows a further update to the soft tissue model 600 used in the dynamic virtual ultrasound where the effect of executing the position based dynamics solver is shown. The position based dynamics solver converges on a solution that best satisfies both the constraints in the soft tissue model 600, such as the constraint 615, and the tissue needle constraints 633, 635, 643, 645 and 653. The result is that the soft tissue of the soft tissue model 600 has been deformed, when compared to Figure 6B, as the particles of the soft tissue model 600 are moved by constraints with the needle 620.

[0115] Examples of a dynamic virtual ultrasound output with a needle will now be described in relation to Figures 7A, 7B, 7C, 7D, 7E and 7F. Each of the figures has a simulated CT scan image shown on the left and a simulated ultrasound image shown on the right. Figure 7A shows a needle 702 touching the skin, as seen in simulated CT image 700 and simulated ultrasound image 705. The soft tissue model is generated using a CT scan of the neck of a patient with the simulated CT image 700 being a reconstructed slice through the CT data, taking into account any deformations made to the model. The simulated ultrasound image 705 is a simulated ultrasound image produced from the simulated CT image 700, with edges enhanced and noise reduced to allow anatomical features to be easily identified.

[0116] A simulated CT image 710 and simulated ultrasound image 715 of Figure 7B show the needle 702 has advanced when compared to Figure 7A. As the needle moves, friction between the soft tissue and the needle 702 drags the soft tissue. The deformation is most evident at the skin where an indentation has been made. The deformation is caused by constraints between the needle particles and the soft tissue model particles.

[0117] A simulated CT image 720 and a simulated ultrasound image 725 are shown in Figure 7C where the needle 702 has advanced further. The needle 702 has entered the Sternocleidomastoid muscle and deformation at the sternocleidomastoid surface is apparent.

[0118] A sequence of simulated CT images 730, 740 and 750 as well as a simulated sequence of ultrasound images 735, 745 and 755 are shown in Figures 7D, E and F where the needle 702 advancement is stopped and the deformed soft tissue returns approximately to an undeformed state. This may be seen at the skin where the indentation is reversed in Figure 7F. [0119] Examples of a dynamic virtual ultrasound output with an ultrasound probe 802, represented by a horizontal line showing a leading edge of the ultrasound probe 802, will now be described in relation to Figures 8A, 8B, 8C and 8D. Each of the figures has a simulated CT scan image shown on the left and a simulated ultrasound image shown on the right. Figure 8 A shows a simulated CT image 800 and a simulated ultrasound image 805. The source of the soft tissue model is a CT of the neck of a patient. A soft tissue model is constructed using mechanical properties derived from the CT scan data. In the soft tissue model used to generate the images of Figures 8A, 8B, 8C and 8D the soft tissue is non- compressible and volume preserving, while regions of air are compressible. The simulated CT image 800 is a reconstructed slice through the soft tissue model, taking into account the deformations made to the model by the ultrasound probe 802. The simulated ultrasound image 805 is a simulated ultrasound image produced from the simulated CT image 800, with edges enhanced and noise reduced to allow anatomical features to be easily identified. Note that normally only an area directly under a transducer of the ultrasound probe 802 would be displayed as output of the dynamic virtual ultrasound.

[0120] Figure 8B shows the ultrasound probe 802 placed against the skin with light pressure in a simulated CT image 810 and a simulated ultrasound image 815. The ultrasound probe 802 flattens the skin and makes good contact between the ultrasound probe 802 and the skin. Soft tissue layers between the ultrasound probe 802 and the trachea, the black oval below the ultrasound probe 802, have not compressed, whereas the trachea, which contains air, has compressed slightly. The different properties of the soft tissue layers and the trachea are a result of different constraints between particles in the soft tissue model.

[0121] Figure 8C shows the ultrasound probe 802 pressed against the skin with more pressure, compared to Figure 8B, in simulated CT image 820 and simulated ultrasound image 825. The trachea has compressed more than shown in Figure 8B while the soft tissue has not compressed. The effect of volume preservation due to the constraints used in the soft tissue regions of the soft tissue model may be seen where the skin of the virtual patient, on the left of the image, is pushed outwards by the ultrasound probe 802. The sternocleidomastoid muscle to the left of the trachea can be seen to be deformed as the tissue displaced by the probe has applied pressure.

[0122] Figure 8D shows the ultrasound probe 802 pressed against the skin of the virtual patient with a heavy pressure in a simulated CT image 830 and a simulated ultrasound image 835. The images of Figure 8D show the trachea has compressed further while the soft tissue has not compressed due to the different constraints used in different regions of the soft tissue model from which the simulated CT image 830 and the simulated ultrasound image 835 were generated.

[0123] A dynamic virtual ultrasound method 900 will now be described with reference to Figure 9. The dynamic virtual ultrasound method 900 may be practiced on a computer such as the processing system 100 communicating over the network 202 and may implement the dynamic virtual ultrasound 300. The dynamic virtual ultrasound method 900 starts with a receive soft tissue model step 910 where a soft tissue model for a virtual patient is loaded to the processing system 100. The soft tissue model may be loaded from a storage device attached to the processing system 100. The soft tissue model is typically prepared before operation of the dynamic virtual ultrasound.

[0124] At a receive medical tool model step 920 one or more models of a medical tool may be loaded by the dynamic virtual ultrasound method 900. The medical tool may be selected by an operator through menu options on the dynamic virtual ultrasound. Alternatively, the medical tool may be selected by having an operator of the dynamic virtual ultrasound pick up a virtual tool in a virtual environment.

[0125] At an add constraints step 930 constraints are added between particles of the medical tool from the receive medical tool model step 920 and the particles of the soft tissue model from the receive soft tissue model step 910. The linking between the particles are described above.

[0126] At an update soft tissue model step 940 the effect of the interaction between the medical tool and the soft tissue model is determined using the constraints added in the add constraints step 930. The updated soft tissue model is determined using a position based dynamics solver as described in relation to the dynamic position based dynamics solver 320.

[0127] The last step of dynamic virtual ultrasound method 900 is a generated simulated scan step 950 where the acoustic simulator 340 described above is used to generate an image of the scanned soft tissue model. The generated image is then displayed to a user of the dynamic virtual ultrasound.

[0128] The configuration of the soft tissue model may be varied for a dynamic virtual ultrasound. For example, spacing between particles may be varied or the arrangement of the particles may be varied by using a different mesh type. For example, the mesh may be square, hexahedral, tetrahedral, or a non-uniform mesh type. A multi resolution mesh may also be used where resolution of the mesh is varied for different regions of the soft tissue model. In one example, the spacing between the particles of the soft tissue model are varied across the soft tissue model so that particles of the soft tissue model are closer together for regions where there is interaction between a medical tool and the soft tissue. Alternatively, the spacing of the particles may vary according to a region type with an air filled region having particles further apart than a less compressible region of the soft tissue model.

[0129] The dynamic virtual ultrasound may use an iterative solver for the position based dynamics solver. Performing more iterations of the position based dynamics solver may provide a more accurate solution at the cost of a longer compute time. In order to meet real time requirements, fewer iterations can be performed at the expense of the accuracy of the physical simulation. In practical terms, so long as the simulation appears to be plausible to the user, and dynamic virtual ultrasound achieves the desired training outcomes or transfer of knowledge, then the accuracy may be considered acceptable.

[0130] The acoustic simulation, as described in relation to the acoustic simulator 340, may use stochastic ray tracing where multiple rays are traced from each transducer element. Tracing more rays may provide a more accurate image with less noise. However, in practice, the details lost by tracing fewer rays may only have a minor effect on the interpretation of ultrasound images while noise may be reduced with post processing.

[0131] The use of a position based dynamics solver in the dynamic virtual ultrasound provides advantages over other approaches to modelling deformation of soft tissue. Approaches such as Finite Element Method (FEM), Mass Spring Damper System (MSDS), or some minor variation may present challenges if used for a dynamic virtual ultrasound. For example, finite element modelling, while being accurate, is a computationally expensive approach that may be too slow for use in a real-time system. Finite element modelling and mass spring damper system approaches may also have difficulty with volume preservation used for incompressible tissue. Further, medical tools require additional constraints that may not easily be represented in a finite element model. Both finite element models and mass spring damper systems operate based on forces, solving for resulting velocities and positions of elements in the system. However, force based methods may become unstable, resulting in large forces, velocities and displacements. The use of a position based dynamics solver, such as XPBD or Small Steps for XPBD, allows the dynamic virtual ultrasound to operate in real-time with a sufficient resolution and stability to be useful as a realistic ultrasound simulation.

[0132] The use of soft tissue models generated from CT scans or MRI scans allows for a broader range of patient types to be used. The range of patients may often be limited by the difficulty in generating a new patient model. The dynamic virtual ultrasound may have new patient models developed from existing scans and may allow for a broad range of patients compared to other systems.

[0133] The interaction between particles of a medical tool and soft tissue model particles allows the soft tissue to push back on the tools. Further, the use of different constraint types allows the soft tissue model to accurately model soft tissue with different properties, such as compressible and incompressible. Different aspects of the soft tissue model may be varied to change operation of the dynamic virtual ultrasound. For example, the dynamic virtual ultrasound may vary the complexity of the soft tissue model depending on what is important for a given simulation. The soft tissue model complexity may be varied by changing the spacing between particles of the soft tissue model.

[0134] The reference in this specification to any prior publication (or information derived from the prior publication), or to any matter which is known, is not, and should not be taken as an acknowledgment or admission or any form of suggestion that the prior publication (or information derived from the prior publication) or known matter forms part of the common general knowledge in the field of endeavour to which this specification relates.

[0135] Throughout this specification and the claims which follow, unless the context requires otherwise, the word "comprise", and variations such as "comprises" or "comprising", will be understood to imply the inclusion of a stated integer or step or group of integers or steps but not the exclusion of any other integer or step or group of integers or steps.