FIELD

[0001] The devices, systems and methods described here are in the field of miniature remote sensing devices, and more specifically, in the field of remote sensing capsules for in vivo inspection of the interior of a body cavity such as the gastrointestinal tract.

CROSS-REFERENCES TO RELATED INVENTIONS

[0002] The present invention is related to the following U.S. Patent Application: U.S.

Provisional Patent Application Serial No. 60/620,778.

BACKGROUND

[0003] It is often necessary to inspect the inside of enclosed spaces, cavities and containers such as reservoirs, fuel tanks, and pipe lines. This generally requires that a camera or other sensor be inserted into the enclosed space and the images or readings taken by the sensor be transmitted and displayed on a suitable device. In particular, it is beneficial to inspect the interior regions of a body cavity such as the gastrointestinal tract, since inspection may help in diagnosing and treating disorders. For example, a variety of disease processes affect the stomach, including infection (Helicobacter pylori), inflammation (gastritis), hemorrhage (ulcer, varices, arteriovenous malformations), congenital aberrations (duplication cysts, pancreatic rests), and benign and malignant tumors (leiomyomata, lipomas, sarcomas, hyperplastic and tubular adenomas, lymphoma and carcinoma).

[0004] The two methods most often used to evaluate the stomachs of individuals with digestive disease are contrast radiography ("upper GI") and esophagogastroduodenoscopy ("EGD" or "endoscopy"). Endoscopy involves inserting a flexible scope though the mouth, along the esophagus and into the stomach. An endoscope typically contains a charge-coupled device (CCD) and a light source, as well as a channel that permits the sampling of tissue from the lining of the stomach (biopsy). Endoscopy has become an essential part of the everyday practice of medicine. Unfortunately, standard endoscopy also has shortcomings that limit its use. For example endoscopy requires that the subject be sedated. Inadequate sedation can lead to an unnecessarily painful procedure, and excessive sedation can lead to a dangerous fall in

blood pressure, oxygen saturation or cardiorespiratory arrest. Also, endoscopy must be performed in a clinical setting such as a hospital or doctors office. Finally, endoscopy is somewhat invasive and requires a great deal of skill and training to perform.

[0005] Recently, wireless endoscopes have been introduced to aid in medical diagnosis.

Such endoscopes typically take the form of capsules that can be ingested by a subject and may be used to take images as they pass through the subject. The first such device was developed and successfully used to investigate the human small intestine in patients with obscure gastrointestinal bleeding [Gastrointest Endosc 2000;51:725-729; N Engl J Med 2001;344:232- 233]. This small bowel imaging capsule had only a single imaging element, a battery, and a telemetry unit capable of transmitting images at a rate of two per second to a receiver worn as a harness by the patient. The device moved passively through the small intestine at various rates of speed, sometimes stopping for hours.

[0006] The first randomized trial comparing wireless capsule endoscopy with endoscopy of the small intestine concluded that wireless technology detected more abnormalities than endoscopy [Gastroenterology 2000;l 19:1431-1438], even using the imperfect wireless endoscopes currently available. The study was conducted in an animal model (canine small intestine) in which 9 to 13 color beads (3-6 mm diameter) were sewn. After recovery from surgery, the dogs underwent wireless capsule endoscopy and standard small bowel enteroscopy. The number, order and color of beads were studied. The investigators conducting enteroscopy and capsule endoscopy were blinded to the surgical placement of the beads as well as each others' results. The sensitivity of the capsule was 64% compared with 37% for enteroscopy (p<0.001). The study also suggests that even the most basic capsule technology can improve upon the limitations of standard endoscopic technology.

[0007] To overcome the slow transit time and passive movement of endoscopy capsules, various experimental approaches to capsule propulsion have been proposed, include electrical stimulation of the small intestine to provoke muscular contractions that can propel the capsule forward [Gastrointest Endosc 2001;54:79-83], and magnetodynamic propulsion systems (e.g., U.S. 6,939,290 to Iddan). Furthermore, existing capsule designs have not been optimized with respect to power (e.g., batter power), imaging, resolution, and user interface.

[0008] Thus, there is a need for a wireless endocapsule that accurately and controllably senses features of a liquid medium such as a subject's gastrointestinal tract. The devices, methods and systems described herein address this need, and the problems described above.

SUMMARY

[0009] Described herein are remote sensing capsule for sensing features within a fluid medium, systems including remote sensing capsules, and methods of using them.

[0010] In general, remote sensing capsules use wireless telemetry to transmit images from cameras (e.g., CMOS elements) incorporated into the capsule. The endocapsule may also contain one or more biosensors capable of detecting changes in pH, the presence of infection or the effect of medications taken by the patient. The ability to assess specific therapies by the physician can permit more careful patient monitoring and more effectively alterations or adjustments in the therapeutic plan. The sensing capsules may also be used without sedation that would be required for traditional endoscopic procedures, making the procedure available to all subjects without regard to their degree of underlying illness or sensitivity to sedatives. The majority of the lining of the stomach could be assessed in "real time" by virtue of the propulsive mechanism described below. This would enable the operator to direct attention to areas of interest, potential areas of disease or other abnormalities.

[0011] In one variation, the remote sensing capsule for sensing features within a fluid medium comprises a capsule body, at least one sensor, and at least one acoustic stream generator for propelling the capsule within the fluid medium. The sensing capsule may also include a power supply within the capsule body, and/or a wireless telemetry system for communicating signals from the sensor with a receiver. In some variations, the sensing capsule also includes a ballast control system.

[0012] The sensor for use with the sensing capsule may be selected from the group consisting of: an optical sensor, a chemical sensor, a temperature sensor, and a pH sensor. In some variations, the sensor comprises a pair of cameras configured to provide stereoscopic video images. The sensor may be configured to function in real time. For example, when the sensor is configured as a pair of cameras for providing stereoscopic video images, the cameras may be configured to provide stereoscopic video images in real time.

[0013] The sensing capsule may also include a multiplexer for processing signals from the sensor (or from multiple sensors). The sensing capsule may also include one or more lights. For example, the capsule may include six lights arranged around the camera or cameras to illuminate features within the fluid medium.

[0014] In some variations, the sensing capsule includes a plurality of sources of propulsion for steering the capsule. For example, the capsule may include one or more thrusters configured as acoustic stream generators. As described below, acoustic stream generators include piezoelectric devices for generating thrust within the fluid medium.

[0015] The sensing capsule may be configured for use within a subject's body. For example, the sensing capsule may be biocompatible, or may have an outer covering that is biocompatible, and may be sterile or sterilizable. In some variations, the capsule is dimensioned so as to fit within a subject's body (e.g., within a subjects gastrointestinal tract).

[0016] Also described herein are remote sensing capsules for sensing features within a fluid medium that inlcude a capsule body, a pair of cameras configured to provide stereoscopic images, and a wireless telemetry system for sending the stereoscopic images. The sensing capsule may be configured to provide stereoscopic images in real time. In some variations, the cameras comprise CCD cameras.

[0017] Sensing capsules may also include one or more power supplies (e.g., batteries), and a telemetry system. The telemetry system may include at least one antenna (and in some variations, only one antenna), and a transmitter and receiver (or transceiver). Thus, the sensing capsule may comprise a single transmitter and a single antenna located within the capsule body for sending and receiving signals. The sensing capsule may also include a multiplexer configured to receive the stereoscopic video images from the pair of cameras before sending the images by the wireless telemetry system. As described above, the sensing capsule may include at least one light source.

[0018] The capsule body may comprise a cylindrical shape having a diameter of less than about 75 mm and a length of less than about 175 mm. In some variations, the capsule body comprises a cylindrical shape having a diameter of less than about 12 mm and a length of less than about 32 mm. The capsule body may be miniaturized even further.

[0019] In some variations, the sensing capsule includes a propulsion system, such as at least one acoustic stream generator. The propulsion system may be an arrangement of acoustic stream generator arranged over at least a region of the external surface of the sensing capsule.

[0020] In one variation, the remote sensing capsule for sensing features within a fluid medium comprises a capsule body, a pair of cameras configured to provide stereoscopic video images, and at least one acoustic stream generator for propelling the capsule within the fluid medium.

[0021] Also described herein are remote sensing systems for sensing features within a fluid medium comprising a remote sensing capsule and a receiver for receiving the stereoscopic video images. The remote sensing capsule may be any of the remote sensing capsules described, including remote sensing capsules comprising a capsule body, a pair of cameras configured to provide stereoscopic video images, and a wireless telemetry system sending the stereoscopic video images.

[0022] In some variations, the remote sensing system includes a processor for processing the stereoscopic video images. The remote sensing system may also include a display for displaying the stereoscopic video images.

[0023] The remote sensing system may also include a navigation controller for controlling the movement of the capsule within the fluid medium. For example, the navigation controller may include one or more control inputs for receiving user input (e.g., a joystick, keyboard, button, audio input, petal, etc.).

[0024] The remote sensing system may be configured for use within a subject's body.

For example, the receiver may be a telemetery belt configured to be worn by the subject. The telemetry belt can communicate with a processor or controller (e.g., a computer) for instructing the sensing capsule. Thus the system may also include a controller for controlling the sensing capsule. The controller may include control logic for steering and guiding the sensing capsule. The controller may also comprise signal processing logic for processing signals from the sensing capsule.

[0025] Also described are methods of sensing features within a fluid medium. The method may include the steps of: placing a sensing capsule into the fluid medium (wherein the

sensing capsule comprises a sensor and at least one acoustic stream generator), moving the sensing capsule by activating an acoustic stream generator, and sensing a feature from within the fluid medium. The method may also include the step of placing a sensing capsule into the fluid medium. For example, the sensing capsule may be placed within a subject's gastrointestinal tract.

[0026] The step of sensing a feature from within the fluid medium may also include capturing real-time video of features within the fluid medium (including capturing real-time stereoscopic video images). In some variations, the step of moving the sensing capsule further comprises a step of signaling (via wireless telemetry) control directions to the sensing capsule.

[0027] Also described herein are improved imaging and sensing systems, and improved propulsion and guidance systems for moving small bodies within a fluid medium. The improved imaging systems and improved propulsion and guidance systems may be used with any of the remote sensing capsules described herein, or with other appropriate devices. In some variations, remote sensing capsules may be used to aid in the clinical diagnosis or investigation of a subject's stomach. Thus, the majority of examples described herein concern wireless gastric endocapsules. As described further below, however, the devices and systems described herein are not limited to gastric endocapsules, and any of these devices and systems may be applied to any other use, including non-gastric or non-biological uses.

[0028] The wireless sensing capsules could also be used in a variety of non-medical arenas (space, under sea, battlefield, site of trauma, etc.) where such advanced and essential technology was previously not available.

BRIEF DESCRIPTION OF THE DRAWINGS

[0029] FIG. 1 depicts one variation of a sensing capsule.

[0030] FIG. 2 shows one variation of a system including a sensing capsule for sensing features within a fluid medium.

[0031] FIG. 3 shows a schematic of a multiplexing system for combining the stereo images and a display using shutter glasses, as described herein.

[0032] FIG. 4 shows an arrangement of a lens, a sensor and the field of view of the sensor.

[0033] FIG. 5 shows the light path and positions of various points within the field of view of the sensor.

[0034] FIG. 6 illustrates the uncorrected image of a surface 5On a sensor using a lens.

[0035] FIG. 7 illustrates the correspondence between a point P on plane 5On a sensor when viewed through a lens.

[0036] FIG. 8 shows the effect of lens alignment.

[0037] FIG. 9 shows the change in the path of a light ray when the lens is displaced along the z axis.

[0038] FIG. 10 shows the effect of lens misalignments in the xy axis.

[0039] FIG. 11 shows a comparison of actual and simulated images of a checkerboard sheet of paper.

[0040] FIG. 12 illustrates the width of the filed of view seen by two cameras of a stereoimaging system at a virtual display plane.

[0041] FIG. 13 illustrates the depth effects of stereo display of images.

[0042] FIG. 14 illustrates the closest distance between an object and the cameras contained in a 10 mm diameter cylinder capped by a hemispherical dome.

[0043] FIG. 15 shows a schematic of a multiplexer.

[0044] FIGS. 16A-16C illustrate switching from a right camera to a left camera by a video switch as described herein.

[0045] FIGS. 17A-17C illustrate one method of tagging video streams from right and left cameras.

[0046] FIG. 18 illustrates a method of calculating the irradiance at pixel P " ' on the sensor.

[0047] FIGS. 19A-19D show changes in the path of the chief light ray during different conditions of alignment and distortion of a lens.

[0048] FIG. 20 shows a method of correcting fisheye distortion and cropping.

[0049] FIG. 21 shows one variation of a sensing capsule as described herein.

[0050] FIG. 22 shows a schematic of a camera circuit.

[0051] FIGS. 23 A-23D illustrate different states of a video switch.

[0052] FIG. 24 illustrates a microcontroller as described herein.

[0053] FIG. 25 A shows a microcontroller controlling switching of a video switch.

[0054] FIG. 25B shows one variation of the logic used to insert tags into a multiplexed video stream.

[0055] FIG. 26 shows a sheet of white paper being illuminated by one, two and six lights.

[0056] FIG. 27 illustrates steps for displaying stereoimages as described herein.

[0057] FIG. 28 shows arrangements of either 1, 2, or 6 LEDs inside a sensing capsule in which the diameter of the PCB is 46 mm.

[0058] FIG. 29 shows illumination along the centerline of the illuminated area for one, two or six lights arranged as shown in Figure 28, where the illumination is normalized with respect to the value at the center of the image.

[0059] FIG. 30 shows a schematic view of one variation of an acoustic stream generator.

DETAILED DESCRIPTION

[0060] Described herein are remote sensing capsule for sensing features within a fluid medium, systems including remote sensing capsules, and methods of using them. The remote

sensing capsules may include a capsule body, one or more sensors, a power supply, a telemetry system, a propulsion system, a buoyancy system, and control systems. Any of these components may be included, and additional components may be included. These components may be at least partly enclosed within the capsule body of the sensing capsule.

[0061] Any appropriate sensor may be used, including optical sensors (e.g., cameras, IR sensors, UV sensors, fluorescence detectors, etc), chemical sensors (e.g., urea detectors, analyte detectors, etc), pH sensors, temperature sensors, electrical sensors (e.g., electrochemical sensors), pressure sensors, etc. Various examples of sensors are provided below. In some variations more than one sensor, including sensors of different types, may be used. The sensors may be configured for continuous detection and transmission of information, or they may be controlled (e.g., by on-board sensor logic or remotely) to collect and/or transmit information over certain time intervals, or after being triggered by some stimulus. In some variations, the sensor is controlled by a practitioner (e.g., a doctor, technician or other user) sending a signal or command to the sensing capsule.

[0062] In general, the sensing capsules described herein operate within a fluid medium.

The fluid medium is typically a liquid medium (e.g., water, biological fluids, etc.) which allows the sensing capsule to navigate within the fluid medium. A sensing capsules may also be referred to as an endocapsule, however it should be clear that sensing capsules may be used for any appropriate application and are not limited to biological (e.g., endoscopic) applications. Sensing capsules may be used to examine any enclosed space containing a fluid medium, such as fuel tanks, pipe lines, wells, springs, etc.

[0063] Any appropriate power supply or power source may be used to power the sensing capsule. For example, battery power may be provided by including an appropriate battery within the capsule body. Batteries may be small (e.g., miniature), and can be rechargeable. For example, a lithium manganese dioxide battery, a silver oxide battery, or a zinc-air battery may be used. Smaller, higher-power batteries may be particularly useful for powering the sensing capsule. In some variations, power may be applied from an external source (e.g., by induction).

[0064] In general, the sensing capsule is completely wireless, and does not tangibly connect to a device outside of the liquid medium in which it can operate. Thus, information sent and received by the sensing capsule is transmitted wirelessly (e.g., via wireless telemetry) so that

it can be detected and relayed, stored, analyzed, displayed or presented. The sensing capsule may therefore include components necessary or helpful for wireless communication, including one or more antenna, multiplexers, transmitters, receivers, and the like. In some variations, one or more intermediate signal transceivers may also be used both within the capsule body, and external to the capsule body. For example, a transceiver may be located external to the fluid medium but sufficiently close to the sensing capsule so that it can relay signals to/from the signal capsule to a monitoring, recording or controlling station. In some variations, a series of transceiver relays may be used. The transceivers may be wireless or wired, and may be of an appropriate shape or power to receive even faint signals from the sensing capsule. For example, a transceiver "belt" may be worn by a subject that has ingested the sensing capsule. In some variations, relays may also modify or record signals (e.g., filter or otherwise process the signals) sent by the sensing capsule.

[0065] As described further below, any appropriate propulsion system can be used with the devices described herein. The propulsion system typically allows the capsule to navigate and access the volume of the fluid medium (e.g., when the endocapsule is used within the GI, the volume may be equivalent to regions such as the stomach). Thus, the propulsion system may include components controlling the propulsion, steering and/or guidance. In some variations a series of transducers producing acoustic streams (also referred to as acoustic stream generators or simply "thrusters") can project sound energy (or vibrational energy) into the fluid medium to provide the sensing capsule with thrust. Steering may be provided by selectively activating acoustic stream generators in different locations around the sensing capsule.

[0066] A buoyancy system (also referred to as a buoyancy control system) may also be included, or may be included as part of the propulsion system and/or guidance system. For example, a buoyancy system may comprise small pistons to control buoyancy by evacuating or filling ballast chambers. The buoyancy control system may be integrated with a control system. For example, the buoyancy of the sensing capsule may be controlled automatically by the capsule guidance system. Thus, if the capsule is steered in a direction requiring an increase or decrease in buoyancy, the control system may coordinate the buoyancy of the sensing capsule without requiring specific commands from a practitioner.

[0067] A control system may control any or all of the components of the sensing capsule, and may receive and process user commands. In some variations, a control system (or a part of a

control system) is located within the sensing capsule body. For example, the capsule may include circuitry and logic (including software and/or hardware and combinations thereof) that make up the control system. The control system may control any of the components of the sensing capsule. For example, a single (master) control system may coordinate and control the sensing system (e.g., the camera(s), biosensors, etc.), the propulsion system (e.g., the acoustic stream generator(s)), the guidance system, the buoyancy system, the telemetry system, the power control, etc. In some variations, multiple control systems may be used. For example, separate control systems may independently control components (or combinations of components) of the sensing capsule.

[0068] FIG. 1 shows a cross-sectional view of one embodiment of a sensing capsule.

This sensing capsule 10 includes a capsule body 20 surrounding some of the components of the sensing capsule, including a battery 30, a transceiver 40 (for sending and receiving data or control information), acoustic stream generators 50 (arranged along one end of the capsule), electronics (e.g., for receiving and processing data and commands and/or for controlling various components of the sensing capsule), a vision system including two cameras 70, 70' (e.g., for stereoimaging) and two lights 75, 75' (shown as LEDs), and chemical sensors 80 (located at least partly on a surface of the sensing capsule that contacts the fluid medium). These features will be described in further detail below. FIG. 1 does not show with any detail the connections or arrangements of these components, which may connected and arranged in any appropriate fashion.

[0069] In some variations of the sensing capsules described herein, the sensing capsule is configured for use within a subject's body. As used herein, a subject may include any animal (including humans) that could benefit from the use of a sensing capsule to examine an internal body lumen. A subject may include a patient, including a patient undergoing a therapeutic and/or diagnostic treatment. Sensing capsules may be ingested (e.g., by swallowing) or inserted (e.g., by a delivery device) into a subject's body. In some variations the sensing capsule may is swallowed or delivered in a fluid medium (e.g., water). The sensing capsule may therefore be sterile and biocompatible. For example, the sensing capsule may be sealed, and at least the outer portion (the region of the sensing capsule exposed to the fluid medium) may be sterilizable.

[0070] As described more fully below, the sensing capsule may also be part of a system for sensing features within a fluid medium. This system may include a sensing capsule, a

receiver and/or transmitter (e.g., a transceiver), user inputs (e.g., joystick, keyboard, etc) for guiding or controlling the sensing capsule, and a display system (e.g., video, audio, etc), or a data storage system. FIG. 2 shows a schematic illustrating one variation of a system.

[0071] In FIG. 2, the sensing capsule 201 is located within the gastrointestinal system of a subject 202 (shown as the subject's stomach 205). The subject wears a transceiver belt 225 that receives information (e.g., video, etc.) from the sensing capsule 201, and relays this information on to a computer 215, including a display (monitor 217). The jagged arrow 230 from the transceiver belt 225 to the computer 215 represents the signal(s) relayed to and from the sensing capsule 201 and the computer 215 by the transceiver belt 225. Although FIG. 2 shows a wireless connection between the transceiver belt 225 and the computer 215, any connection may be used. For example, the transceiver belt may be tangibly connected (e.g., wired by cable, etc.) to the computer, or it may be wireless connected. Furthermore, a transceiver belt may not be used. In some variations, more than one transceiver may be used. For example, one or more relay transducers may be included as part of the system. These relays may be located externally to the fluid medium, as shown by the transceiver belt 225 in FIG. 2. In some variations, a relay transducer may be included with the sensing capsule in the fluid medium. For example, one or more relay transducers may be included within the sensing medium when the sensing capsule is used to examine non-biological volumes, such as pipelines, tanks, etc.

Capsule body

[0072] The capsule body may be any appropriate size or shape. In general, it is desirable that the capsule be "miniaturized." That is, the capsule is typically small enough to fit into the small regions of the bodies into which the sensing capsule are used. For example, in variations adapted to be used within a subject's body, the sensing capsule can be small enough to be swallowed or otherwise ingested, and small enough so that it can be navigated through at least a portion of the gastrointestinal tract (e.g., stomach). In many applications, a smaller size is beneficial to operation of the device. Thus, when the capsule body is generally cylindrically shaped, the device may be less than about 75 mm in diameter and have a length of less than about 175 mm. More preferably, the device has a diameter of less than about 12 mm and a length of less than about 32 mm. In one variation, the sensing capsule is a rounded cylindrical shape having a length of about 30 mm and a diameter of about 10 mm.

[0073] Although most of the examples described herein show capsule bodies that are cylindrically-shaped, any appropriate shape may be used. In general, it may be beneficial for the capsule body to be rounded or blunt, to avoid damage to the walls of the fluid medium (particularly when the sensing capsule is used in biological applications). The capsule body may also be substantially smooth. Other variations of the capsule shape include round and elliptical. However, in some applications it may be desirable to have the capsule body be pointed, or otherwise include an irregular or rough shape. For example, the capsule body may have a cross- section that is rectangular, square, etc.

[0074] The capsule body may be shaped to assist guidance and control of the sensing capsule. For example, in some variations, the capsule body contains fins, channels, etc. to help control the flow of fluid over the capsule body. The shape of the capsule body may also be adapted to aid in propulsion of the sensing capsule due to the propulsion system. For example, the acoustic stream generators may be arranged across the surface of the capsule body to guide the motion and direction of the sensing capsule. The capsule body may also include one or passages or chambers. Ballast chambers may be formed within the capsule body, and the passage of fluid from the fluid medium may be regulated into or out of these ballast chambers through the capsule body.

[0075] In some variations, the capsule body (including the outer wall of the capsule body) comprises different regions. For example, the capsule body may include a transparent or lens region allowing imaging or sensing through the capsule body wall, and translucent or opaque regions. In some variations, regions of the capsule body may be adapted as sensors (e.g., electrodes, membranes, etc.). Regions of the capsule body wall may also be adapted as transducers (e.g., the acoustic stream generators providing thrust).

[0076] In some variations, the capsule wall is unbroken or smooth. For example, the xapusle may be enclosed so that there a not any passages through the capsule open to the fluid medium, particularly passages forming fluid pathways completely through the capsule body.

[0077] The interior region of the capsule body may also comprise different regions.

Thus, the capsule body may include regions that are sealed to protect them from interaction with other regions or components of the sensing capsule or from the fluid medium. For example, the

battery may be kept in a sealed compartment, with electrical connections linking it to other components of the capsule body.

Sensors

[0078] Any appropriate sensor may be used as part of the sensing capsule, including cameras, as described more fully below under the section titled "Imaging system." Sensors may include optical sensors, chemical sensors, pH sensors, temperature sensors, electrical sensors, pressure sensors, and the like. Although various examples of specific types of sensors and ways in which they may be successfully used as part of a sensing capsule are described herein, it should be understood that sensing capsules are not limited to the sensors described.

[0079] Optical sensors may include the imaging sensors described below, as well as optical sensors that detect intensity (e.g., photon counting), and/or the presence/absence of photons, or of specific wavelengths of light. For example, a sensor may comprise an IR sensor, a UV sensor, a fluorescence detector, etc. These different types of sensors may be combined. For example, a CCD camera may be used to image as well as to determine intensity. An optical sensor may be used in conjunction with a light source (e.g., an LED). A light source may emit a characteristic wavelength of light that may excite a response from a target material (which then fluoresces or emits at a target wavelength) that can be detected by the optical sensor.

[0080] Chemical sensors may be used to detect various chemical compositions (e.g., urea detectors, analyte detectors, etc.). For example, a chemical sensor may be a redox-mediated chemical sensor that detects a compound by enzymatically oxidizing or reducing the compound (or a byproduct of the compound). Other examples include the use of fluropolymers that react with certain analytes (e.g., by changing color), or the like. For example, see U.S. 6,653,148, U.S. 6,947,138, U.S. 6,925,213, the contents of which are herein incorporated by reference in their entirety. Chemical sensors may also detect pH. Examples of pH sensors are well known in the art, and include colorimetric and electrolytic detection methods.

[0081] Temperature sensors may be used. Examples of temperature sensors include electronic sensors (e.g., based on thermoelectric properties, such as changes in electrical conductance with temperature, etc.) and colorimetric sensors. In general, the sensing capsule may include one or more electrode pairs for sensing electrical energy. For example, a surface of

the capsule may include one or more electrode pairs to measure electrical properties of the fluid medium (e.g., conductance, ionic concentration, etc.). Electrodes on the sensing capsule may also be used to detect electrical properties of tissues making up the body cavity (for biological applications) or of the container of the fluid medium (for non-biological applications). Additional sensors that may be used include pressure sensors and the like.

[0082] The data received by the sensors may be transmitted in any appropriate fashion.

For example, data from the sensors may be transmitted by digital or analog transmission using the telemetry system of the sensing capsule (or using a dedicated telemetry system). In some variations, the signals from the different sensors may be combined. For example, the sensors may include visual indicators that can be imaged by an imaging system (e.g., the imaging system described below) and transmitted with other visual data. Thus, in some variations, a sensor comprises a region of the capsule body, particularly the window region, and thus can be imaged by an imaging system to transmit the data.

[0083] The sensor may comprise a biosensor for detecting one or more biologically relevant properties. One variation of a biosensor is a urease-detecting biosensor. A urease- detecting biosensor may detect digestion of urea is digested to CO ₂ by Helicobacter pylori (which may indicate infection). Thus, a urease-detector may be incorporated into the capsule. For example, digestion of urea may raise pH in a perceptible manner, and may also result in detectably degradation products.

[0084] Sensors may be in contact with the fluid medium (e.g., chemical sensors) or they may be contained within the capsule body. As mentioned above, the capsule body may include regions that are configured as sensors, or sensors may be mounted to the outside of the capsule body, and viewed or otherwise monitored through the capsule body. In some variations the sensor traverses the capsule body, and may be sealed to prevent undesirable leakage of the fluid medium into other regions of the sensing capsule. In some variations, the sensing capsule may include passages or chambers which may be used to sample material from the fluid medium. Thus, the sensors may be located entirely within the sensing capsule and be used to analyze material sampled from the fluid medium.

[0085] Any of the sensors used as part of the sensing capsule may also be used in combination with additional sensors. Furthermore, the sensors may be used to guide or control

the capsule. Imaging, and particularly real-time imaging, may be used to help guide or control the remote sensing capsule.

Imaging system

[0086] The sensor may also comprise an imaging system for visualizing features of the fluid medium. In particular, the sensing capsules described herein may include stereoscopic imaging. Stereoscopic imaging generally uses two cameras to detect visual information that may include spatial perspective information. Thus, stereoscopic information may be used to analyze or display this enhanced visual information. In the examples provided below, dual CMOS elements are used to provide stereoscopic (e.g., 3D) imaging information. Any appropriate imaging technology may be used (e.g., CCD, CMOS, etc.). Furthermore, the sensing capsules described herein may also include real-time imaging, including real-time stereoscopic imaging. Because it may be desirable to minimize the size of the sensing capsule, and the large amounts of information collected, particularly when imaging dual (or stereo) images, the sensing capsule may include a telemetry system for transmitting the information from a single antenna using a single transceiver.

[0087] Typically, stereoscopic images are constructed from images from two different cameras, such as a right camera (corresponding to a right eye), and a left camera (corresponding to the left eye). The stereo image is displayed by either differently polarizing the images from the left and right cameras or by filtering the images from the two cameras by complementary colors, so that each of a viewers eyes is exposed to the correct image. Although both of these techniques (the colored glass and polarization techniques) may be used to display stereoimages to a viewer, these techniques typically require two transmitters and two antennas when sending images from a wireless sensing capsule. This is particularly true when stereoimaging in real time. Alternatively, a merged video technique may be used, in which images streams from a left camera in the sensing capsule and the right camera in the sensing capsule are merged. This merged video technique results in a merged signal containing alternating frames of images from the left and right cameras. Frames from the left and right cameras may be marked (as described below) to indicate which camera they originated from before they are combined into the merged stream. A method of viewing involving shutter glasses (which alternately display images on the left and right lenses of the glasses) is also described below. Alternatively, the merged signal may be broken into separate signals by the receiver (e.g., a display or control device for

processing the images) and presented as either colored, polarized, or non-stereo images. FIG. 3 shows a schematic illustrating the multiplexing system for combining the images and the display using shutter glasses described above.

[0088] In general, to achieve real-time stereoscopic images from a sensing capsule, the videos from the two cameras are multiplexed into a single stream for transmission. The video is demultiplexed inside a demultiplexer (e.g., a computer), and the images can be processed. For example, the images can be corrected for fisheye distortion and lens misalignment, and cropped to the proper size.

[0089] In one variation, the optical system takes stereoscopic video, transmits the video to a computer, and displays the stereoscopic video on the computer screen. The major components of the system are a capsule, a transceiver, and a computer.

[0090] Typically, two cameras are needed to generate stereoscopic pictures, and the camera lenses should preferably have a wide angle of view in order to see as large an area as possible. The sensors for each lens must be located at the back focal length (bfl) and should be sufficiently large to accommodate the entire image captured by the lens. As shown in FIG. 4, the image that appears on the sensor may be distorted (e.g., fisheye or barrel distortion and lens misalignment) and must be cropped to the required size. Thus, the image taken from each camera may be processed to correct the distortion and to appropriately crop the images. One manner in which these tasks may be performed is described below.

[0091] The spacing and layout of the miniaturized stereoimaging system can be determined as described below. This description serves to illuminate some of the considerations for implementing a stereoimaging system as part of a sensing capsule.

[0092] The light reflected from point P on the surface of an object would be transmitted to pixel PO on the sensor if the lenses were perfect and were correctly aligned, as illustrated in FIG. 5. The coordinates off and PO are given below. The position of point P on the surface of the object is described by the relationships:

Xp = r cos γ, yp = r sin γ, Zp = d, and r = d tan δ

[0093] Where x, y, and z are the coordinates relative to the lens' first nodal point Ni, d is the distance between Ni and the plane S normal to the optical axis and passing through P, the angles δ and γ are defined in FIG. 5, and r is given by:

[0094] In the ~x, ^~ y coordinate system with the origin at the sensor's center O, the location of PO is

[0095] where r ⁰ is the distance between O and PO. The lens is symmetric and the angle γ° equals γ. Due to lens distortion and misalignment of the lens, the image of P is not at PO but at P ' ". To obtain the correct image, the information at P " ' must be transferred to PO-. This process requires that the location of P " ' be known. The coordinates of P " ' and the corresponding PO are obtained in four steps.

[0096] Step 1 : Fisheye (or barrel) distortion causes straight lines on the object to appear as curved lines on the sensor (FIG. 6), and the image of point P to appear at pixel PO (FIG. 7). For a correctly aligned lens the location of PO is (FIG. 7):

[0097] where r ' is the distance between O and PO. For a particular lens the lens manufacturer specifies r ' as a function of r ⁰ and δ.

[0098] Step 2: The distance dNi between the lens and the sensor should have the value specified by the manufacturer, and the optical axis should be perpendicular to the sensor plane and should intercept the sensor at O. In practice, it is difficult to mount the lenses in such a way that all three of these conditions are met, and any misalignment must be corrected. When the optical axis is "tilted" (i.e. it is not perpendicular to the sensor) parts of the image on the sensor become unfocused. To eliminate this problem the position of the lens must be adjusted

mechanically so that the entire image is in focus. The problems caused by displacements of the lens along the z axis and in the x - y plane can be corrected as follows.

[0099] When the lens is displaced by A(N ₁ ) _Z along the z axis the image at pixel P ' appears at pixel P ' ' (FIG. 8). The location of P " is:

[0100] where γ " = γ ' = γ° = γ. From geometry, the distance r " is (FIG. 9):

[0101] where dNj is the distance between the correctly positioned lens and the sensor, and dNj « d.

[0102] Step 3 : When the lens is displaced by A(Ni) _x and Δ(Nι) _y in the x - y plane, the optical axis intersects the sensor at O ' (FIG. 10). The location of O ' in the ~x - ~y coordinate system is:

[0103] Due to the shift of the optical center of the image from O to O ' the image that was at pixel P " is now at pixel P'" located at

[0104] which, when combined with the previous equation, becomes:

[0105] Step 4: The location of PO corresponding to P " ' is determined by the following steps for both the left and right cameras:

[0106] The above calculations require that the distances A(Ni) _x , Δ(Ni) _y and Δ(N _\ ) _Z be known. These distances may be determined by the following procedure. A video is taken of a flat sheet of checkerboard paper (FIG. 11). The video is stopped, and a frame from the right camera is stored. The frame from the right camera is then displayed on the screen without correcting for lens distortion and for lens misalignment. The uncorrected image of the checkerboard paper, as would be "seen" by the right camera, is simulated by calculating the coordinates ~Xp • • • and ~yp "- of points P '", each P"' corresponding to one of the node points P on the checkerboard pattern on the paper. The calculations proceed as follows:

[0107] The uncorrected image thus constructed is displayed on the screen together with the actual image of the checkerboard paper from the right camera and the two images are visually compared. If the locations and the sizes of the checkerboard squares do not match, a

new set of values of A(TV ₁ ) _X , A(TVOy and Δ(TVi) _z is chosen, and a new simulated image is constructed and compared to the actual picture. The procedure is repeated until the actual and simulated images match. The same process is performed with the left camera.

a. Distance Between the Cameras

[0108] The distance between the cameras may also be determined. For example, when a computer screen is used to display the images, a distance A between the two cameras (FIG. 12) must be such that a person sitting in front of the computer screen effortlessly sees the stereoscopic image. To satisfy this objective the distance between the lenses must be:

[0109] The angle ψ is the field of view of each lens, w is the width of the image on the screen (FIG. 13), and Z' is the distance between the camera and the virtual display plane (FIG. 12), given by the expression:

[0110] TV' is the desired closest and F' is the desired farthest distance in the scene from the cameras (FIG. 4), and Q is a parameter defined as:

[0111] Z is the distance from the eyes of the viewer to the display, TV is the farthest distance in front of and F is the farthest distance behind the display where the image of the object may appear (FIG. 13). The parameter d^ is defined as:

[0112] where e is the distance between the viewer's eyes (for an average person e is about 65 mm). Tests have shown that for best picture quality the viewer"s eyes should be about

500 mm from the display (Z = 500 mm) and N and F should be 200 mm and 500 mm, respectively.

[0113] For example, for a 10 mm diameter capsule with a hemispherical dome the closest distance an object could be to the camera is N' = 5 mm (FIG. 14). Let us assume that the farthest distance the camera can be from the wall is about F' = 350 mm. Then, for cameras with ψ = 125° lenses displaying the video on a w = 300 mm wide screen, the required distance between the cameras is A = 4.5 mm.

b. Multiplexing

[0114] A video encoder, built into the camera, generally transmits the output of the sensor as a National Television Standards Committee (NTSC) signal in such a way that in each frame it first transmits the odd lines and then the even lines. The video signal streams from the two cameras are multiplexed such that the two streams are merged. This can be accomplished in different ways. In the field sequential technique only odd fields are transmitted from one of the cameras and only even fields from the other camera. In the over-under technique only every other line of each field is transmitted. In the side-by-side technique each line is compressed to half its width, and the lines from the right and left cameras thus compressed are combined into one line and transmitted as one line.

[0115] Here, we describe a technique where alternate frames from the right and the left cameras are transmitted. In this manner every other frame from each camera is transmitted. In this technique spatial (though not temporal) resolution is maintained. The loss in temporal resolution is acceptable when the scene changes slowly compared to the frame rate of the cameras.

[0116] Multiplexing is accomplished with a video switch, a sync separator, and a microcontroller (FIG. 15). The video streams from the right R and left L cameras enter the video switch. When the video switch is in the R-O position the output is from the right camera, and in the L-O position the output is from the left camera (FIG. 16A-16C). The sync separator senses the start of a new odd field and sends a signal to the microcontroller that, in turn, switches the stream from R-O to L-O or from L-O to R-O, as the case may be. The frames from the different cameras, combined into a single stream of video by the multiplexer, must be separated

in the receiver. To accomplish this, each frame is identified by the microcontroller by tagging the frames originating from the right camera by a black signal and from the left camera by a white signal. The microcontroller has a built in line counter and a timer. The tags are inserted into the 22nd and 23rd lines of the fields because the first 21 lines do not show up in the display computer where the tags are identified. When the counter indicates the 22nd line, the timer starts and the microcontroller switches the video switch to B-O, thereby inserting a black signal into the frame originating from the right camera (FIG. 17A-17c). The timer stops after a predetermined length of time t _e , and the microcontroller sends a signal to the video switch to switch back to R-O. The time t _e during which the black signal is active is such that the tag does not extend along the entire line. For an NTSC video signal the duration of a line is 64 μs _> and a suitable value for t _e is 15 μs. The process is repeated for the next line, i.e. the 23rd line in the field. An identical process is used to insert a white signal into frames originating from the left camera.

c. Lights

[0117] The imaging system generally includes at least one light (e.g., LED). In general, the light (or lights) must be arranged in such a way that the entire area within the field of view of the two cameras is illuminated with light intensity that is equal to or higher than the intensity required by the cameras. The size of the area illuminated with sufficient intensity is determined as follows. We consider a diffusely reflecting flat surface S perpendicular to the optical axis of the camera (FIG. 18). The surface S is at a distance d from the first nodal point Ni of the lens. In photometric units, the illumination Ep (lux) at point P on a diffuse surface S due to a single light is:

[0118] where Ii (candela) is the luminous intensity of a Lambertian light source in the direction normal to the surface S, di is the normal distance from the light to the surface S, and β is the angle between the light ray and the normal to the surface at P.

[0119] The brightness B _P (candela / m ² ) at point P is:

[0120] The illumination Ep" • (lux) at point P'" is:

[0121] where θ is the angle between the optical axis and the line from the center of the sensor to the edge of the exit pupil of the lens and φ '" is the angle between the chief light ray and the normal n ' " to the sensor at pixel P'" (FIG. 19). The angle φ ' " is equal to φ, and the distance r ' " is equal to r ". Thus, we have:

[0122] Thus, Ep'" can be expressed as:

[0123] The expression above gives the illumination on the sensor at one point due to one light. When there is more than one light, the illumination is calculated for each light and the total illumination is the sum of the individual illuminations. The area illuminated is obtained by calculating the illumination on every pixel and noting those pixels that receive the required minimum illumination.

d. Transmitter, Power supply, Transceiver

[0124] The transmitter and the antenna should be small enough to fit inside the capsule, should transmit a signal with sufficient power to enable reception outside the body, and should operate at a high frequency within the unlicensed Industrial, Scientific and Medical (ISM) band. High frequency is desirable to minimize noise and interference from other electrical sources. The power supply should fit into the capsule, have a high capacity as expressed in watt-hour to provide a long operating time, and have a voltage output that is easily transformable to the

voltage requirements of the circuit. The transceiver may transmit the signal to a computer either wirelessly or via a cable.

e. The Display

[0125] When displaying the stereoimages the image may be displayed on a computer and viewed with shutter glasses. In some variations, the video signal is input to a computer where it is demultiplexed and then displayed on the screen for viewing. Demultiplexing identifies the frames coming from the right and left cameras by examining the tag on each frame in the memory as follows.

[0126] The first line of each frame in the computer memory contains the tag. When the pixels in the tag consist of all zeros, it is a black tag and the frame is from the right camera. When in each pixel of the tag all three numbers are equal to 255 (and, consequently, their sum is 765) the tag is white (because the sum of red, green, and blue is white) and the frame is from the left camera (the addresses of the pixels in the memory are provided by the video digitizer).

[0127] Before the images are displayed they are first corrected for fisheye distortion and lens misalignment (FIG. 20). Then those portions of the images are identified that are within the largest inscribable rectangles. The only portions displayed are those that are within the rectangles and are within the fields of view of both cameras. The field of image seen by both cameras must appear at the same location on the display. These requirements are ensured when the optical centers of the right and left images, points O R and O _∑ , are along the same ~x axis and are separated by a distance C _r = crop ^• VtV(I - crop), where w is the width of the display on the screen and crop is defined as:

[0128] where w ' is the width of the image "seen" by both cameras at the virtual display plane and, as before, A is the distance between the two cameras (FIG. 12). From geometry, we have:

[0129] Once the images from the right and left cameras are aligned in the manner described above, the images within the non-shaded rectangle in FIG. 20 are retained and displayed while the area outside is discarded.

[0130] To view these alternating images we use shutter glasses synchronized in such a way that the right lens is opened and the left one is closed when the right image is displayed, and the left lens is opened and the right one is closed when the left image is displayed. The frames arrive in the computer memory at the standard NTSC rate of 29.97 (approximately 30) frames per second. If the frames were displayed on the screen at the rate at which they arrive, 15 frames per second would be displayed from the right camera and 15 frames per second would be displayed from the left camera. Frames displayed at such rate present a flickering image to the viewer. To reduce the flickering to an acceptable level, a frame from the right camera and the subsequent frame from the left camera are stored in the computer. The stored frames from the right and left cameras are alternately displayed on the screen four times. In this manner the right and left frames each appear 60 times per second. At this display speed the rate of flickering is above the upper limit of human perception. When both left and right images are shown on the same monitor, the display rate may be double the rate for each eye (e.g., 120 Hz). Thus, the flicker otherwise associated with NTSC-based stereo video systems is eliminated.

[0131] In some variations, the stereo images may be viewed without shutter glasses. For example, the images may be viewed by color filtering the images or by polarizing the images from the left and right cameras differently. In some variations, the left and right images may be combined into a single view. In some variations the video may also be viewed on individual displays for each eye.

Example 1

[0132] We designed and built a proof-of-concept device based on the principles described above. FIG. 21 shows an example of the device. The capsule contains two circuit boards, one with the video switch, microcontroller and sync separator, the other one with the power supply components. In the construction of these boards and in the selection of all other hardware components, we made an effort to use commercially available parts. The display is presented on a computer screen. The main cpu and video card in the computer runs the software controlling the display.

[0133] The shell of the capsule is a 150 mm long and 50 mm outside diameter plastic cylinder, with wall thickness of 1.5 mm. The capsule is capped on each end with a hemispherical dome, one of which is transparent. There are two camera assemblies inside the capsule. Each camera assembly consists of a lens, a camera chip, an oscillator, resistors, and capacitors (FIG. 22). The lenses are Boowon Optical Company model BW21B-1000. The cameras are installed 20 mm apart from each other. With the parameters listed in Table 1 (below), the relationships described above give the allowable minimum distance between the object and the cameras as N' = 16 mm. This means that any object outside the 16 mm range would appear in stereoscopically on the display. The proof-of-concept capsule has a 50 mm diameter hemispherical dome, and the lenses are located in the base plane of the hemisphere. Thus, for the proof-of-concept capsule, the closest an object can be to the camera is 25 mm. Therefore, the 20 mm separation between lenses is acceptable and indeed yields good quality stereoscopic images.

Table 1 : Values used in calculating the ideal distance between the two cameras Example

[0134] The lenses are attached to a printed circuit board (PCB) with a dual lens mount.

The camera chip is an OmniVision OV7910 model. This camera chip is sensitive to infrared light, and red objects appear brighter than the actual color. For this reason a hot mirror filter is placed in front of each camera chip. The filters used were taken from two Marshall Electronics V-LH4-IR lens mounts, and were reduced in size to 10 mm diameter to fit the dual lens mount used for the proof-of-concept device.

[0135] The camera chip requires a 14.31818 Mhz clock input. To ensure that the two cameras are synchronized, they share a single Citizen model CSX750PCC14.31818 oscillator. In addition, they are also turned on simultaneously; therefore, there is no need for synchronizing signals from the multiplexer to the cameras. The camera circuit is described in more detail in Z. Kirάly, "A stereoscopic vision system, " PhD. dissertation, Stanford University, 2005, which is herein incorporated by reference in its entirety.

[0136] The main components of the multiplexer are the video switch, sync separator, and microcontroller (FIG. 15). An Intersil model HA4314BCA video switch is used that alternatively sends the video from the right and left cameras and tags the frames coming from the right and left cameras. The Intersil switch has four video inputs (labeled InO through In3, FIG. 23), three control inputs (labeled CS, AO, and Al), and one video output. Inputs InO and InI are connected to the right and left cameras, respectively. Inputs In2 and In3 are used to insert the black and white tags. The control inputs CS, AO, and Al receive the signals from the microcontroller. These control signals are in the form of voltages and may have any value between 2 to 5 V (designated as Hi voltage) or between 0 to 0.8 V (designated as Lo voltage). The input voltages to In2 (black tag) and to In3 (white tag) are 0.5 V and 1.3 V, respectively, are obtained from the +5 V power supply through voltage dividers.

[0137] The video sync separator is a National Semiconductor model LMl 881 video sync separator. It senses the output from the video switch and sends a signal to the microcontroller when a new field or a new line starts. The video sync separator also signals the microcontroller whether the new field is odd or even. The video sync separator requires decoupling capacitors and an external timing circuit.

[0138] The Atmel model AT90S8518-8IA microcontroller signals the video switch to switch from one camera to the other and instructs the video switch to insert a black or white tag into the appropriate lines. This microcontroller requires an 8.0 Mhz clock input which is provided by a Citizen model CSX750PCC8.000MT oscillator.

[0139] The inputs to the microcontroller are from the video sync separator and indicate a new line, new field and whether a new field is odd or even (FIG. 24). There are three outputs from the microcontroller that act as control inputs to the video switch. Depending on the input from the video sync separator and on the states of the timer and the line counter inside the

microcontroller, there are four possible combinations of output signals. These, and the resulting positions of the video switch, are shown in FIG. 25 A. The sequence by which the microcontroller signals the video switch to switch from the left to the right camera and to insert the black tag in lines 22 and 23 is given below and is also illustrated in FIG. 25A. FIG. 25B shows one variation of the logic used to insert the tags.

[0140] When the video switch is in the L-O position and the last line (line 525) has finished streaming to the output of the video switch, the video sync separator registers the start of the next field, and that it is an odd field.

• The video sync separator sends the appropriate signals to the microcontroller.

• The microcontroller commands the video switch to change to the R-O position, and, at the same time, resets its line counter to 1.

• The video sync separator sends a signal to the microcontroller at the start of each line. In response, the microcontroller increments the line counter by 1.

• When the line counter reaches 22, the microcontroller instructs the video switch to change to the B-O position (to insert the black tag signal), and also starts an internal timer.

• When the timer indicates that 15 μs has elapsed, the microcontroller instructs the video switch to change back to the R-O position.

[0141] Switching from the right to the left camera and inserting the white tag is performed in an identical manner. The operation of the microcontroller requires the execution of appropriate software that we have written in C and translated into machine language using the AVR-GCC compiler. The software was downloaded to the flash memory of the microcontroller using an Atmel model STK500 evaluation kit and a custom made adapter.

[0142] The OmniVision OV7910 camera chip requires at least 3 lux illumination which are provided by lights built into the capsule. The lights are Ledtronics model LF200CW6K-27 LEDs soldered onto the PCB.

[0143] The number of lights should be selected such that as large an area as possible is illuminated and the illumination is uniform across the visible area. To assess the effects of the

number and positions of the lights, we installed one, two or six lights inside the proof-of-concept capsule, and displayed on the screen the image of a sheet of paper placed 50 mm from the lenses. We make two observations from the resulting images shown in FIG. 24. First, the apparent level of illumination is nearly the same for one, two and six lights. The reason for this is that each camera has a built in automatic brightness control that scales the brightness of the image so that the output signal stays within the NTSC standard limits. This brightness control negates the increased illumination given by more lights as long as the required minimum illumination of 3 lux is present. Second, the size of the illuminated area depends on the distance between the lights. With six lights the lights were spread out further, thereby increasing the illuminated area.

[0144] The transmitter is a RF- Video model SDX-21LP. Power is supplied by a CR2 battery. The nominal output of the battery is 2 V, while the circuit of the proof-of-concept capsule requires both positive and negative 5 V. A Maxim MAX 1765 DC-DC converter and a MAXl 697 inverting charge pump is used to obtain regulated +5 V and -5 V, respectively. The power supply is turned on and off remotely by a magnetic switch that we incorporated into the circuit specified by the manufacturer.

[0145] From the receiver, the signal may be transmitted to the computer on which the video is displayed either wirelessly or via cable. Here, for the proof-of-concept device, the video signal is fed to the computer by an RCA video cable. We used a Wavecom WCJ30-102 model receiver that is recommended to be used with the RF- Video model SDX-21LP transmitter.

[0146] The images were displayed on a display consisting of a computer fitted with, a video digitizer and a video card. The images displayed on the computer monitor are viewed by shutter glasses. The video signal is input into a 1.4 Ghz Pentium 4 class computer through an AverTV video digitizer. This video digitizer comes with driver software; however, instead of this driver we used the WDM Capture Driver because it requires fewer CPU cycles and has more options for the programmer than the original driver. The video digitizer converts the analog video signal into digital format and inputs it into the memory of the computer. The capture driver provides the location of each frame in the memory. The video is then demultiplexed, as follows. Each frame is examined to determine whether or not the frame has a black or white tag, i.e. whether or not the frame is from the right or left camera. A right frame and a consecutive left frame are transferred to a Nvidia Quadro 4 750 XGL video card where each frame is corrected for fϊsheye distortion (as described above) and for lens misalignment, and is cropped.

The video card then displays the images on the computer screen in the proper sequence, and in such a way that the image's width w on the screen is the same aathe width for which the camera placement is designed.

[0147] The images are viewed with eDimensional shutter glasses. The synchronization signals are transmitted to the shutter glasses wirelessly through an infrared emitter, also supplied by eDimensional. The emitter is connected to the stereo signal output of the video card that sends signals with instructions to open and close the appropriate shutters. For convenience, the emitter is placed on top of the monitor (FIG. 27).

[0148] We also developed software for determining the distance between the lenses, the required number of lights, the positions of the lenses, and for controlling the microcontroller and the display. The software for the microcontroller was written in C and the software for demultiplexing (run on the display computer CPU) in C++. The software for correcting the fisheye distortion and the lens misalignment, cropping, and frame speed-up runs on the video card, and was written using the OpenGL interface. The use of OpenGL greatly enhances the speed and thereby making possible the performance of these operations in real time.

[0149] Using the proof-of-concept device we performed tests to assess the validities of the models and the corresponding algorithms. We placed a white flat surface 50 mm from the cameras and illuminated the scene with either one, two or six lights, placed as shown on FIG. 28. We recorded the illumination at each pixel in the memory by taking the vector sum of the red, green and blue color coordinates:

[0150] where, as we recall, each component may have a value between 0 and 255. We also calculated the illumination by the model. Both the measured and calculated illumination were normalized with respect to the illumination measured at the center of the surface, I _c . The illumination along the horizontal centerline is shown in FIG. 29. The comparisons presented show good agreement between the measured and calculated values, and this lends confidence to the model and the algorithms described above.

[0151] Thus, the optical system of Example 1 provides in real-time stereoscopic video images from a sensing capsule. One advantage of this imaging system is that it requires less space than conventional techniques. The proof-of-concept example, built mostly with commercially available hardware, demonstrates that the proposed vision system can be built on a small scale. Further miniaturization is possible. For example, the device could be fabricated from custom made components.

Enhancements

[0152] The imaging system may be modified or enhanced in any appropriate fashion.

For example, the vision system may be configured to image different wavelengths of light, such as ultraviolet or infrared. Optical filters may be used with the vision system to enhance the imaging of different wavelengths or different features. Furthermore, light sources which emit light of different wavelengths may also be used. For example, one or more light sources included with the imaging system may emit light at a wavelength that can cause florescence. For example, the florescence of tissue may change in response to diseases, including tumors and the like. In some variations, one or more markers (e.g., dyes, indicators, etc.) may be added to the fluid medium to enhance imaging by the imaging system of the sensing capsule. For example, a florescent marker for disease or infection may be ingested so that it can be taken up by a pathogen such as bacteria (e.g. Helicobacter pylori), or so that it can highlight various tissue types (such as inflamed tissue, cancerous tissue, etc). Other contrasting agents may also be used.

[0153] As described above, the imaging system may also be used to perform colorimetric analysis. The ability of the operator or analysis device to detect color enhances the spectroscopic evaluation of the image. For example, when visualizing the stomach lining, spectrophotomeric analysis of an image to determine color change (including color change over time or area) may help determine the presence of inflammation or trauma to the stomach lining. Erythema is one hallmark of inflation which may be detected by spectroscopic evaluation.

[0154] As described above, any appropriate signal processing may be performed on the images or other sensor data, either post-hoc, or as the images are being displayed. In some variations, the images are enhanced (e.g., to show spectroscopic, florescent, etc.) in real-time and displayed to a practitioner using the sensing capsule.

Propulsion system

[0155] Any appropriate propulsion system may be used for the sensing capsules described herein. In particular, the sensing capsules may include non-contact or "zero net flow" propulsion systems. Non-contact propulsion systems mean propulsion systems that do not act by propellers, channels, or other surfaces that apply force to either the fluid medium directly or to the walls of the fluid medium (e.g., the lining of the stomach). In general, non-contact propulsion systems apply thrust to move the sensing capsule within the fluid medium.

[0156] One particularly useful non-contact propulsion system is one in which thrust is provided by acoustic streaming produced by an acoustic stream generator (also referred to as a transducer). This system has no moving parts, and allows small vehicles (e.g., a sensing capsule) to operate in sensitive environments such as the human body. Furthermore, these transducers may be miniaturized and may be particularly useful for providing force to move and control small devices.

[0157] Propulsion from the transducers is provided without the use of pumps or impellers (that are typically difficult to miniaturize, may clog or become less effective in impure or inhomogeneous environments, and may be harmful to sensitive biological tissues in the gastrointestinal or cardiovascular system). Instead, the system provides propulsion by producing acoustic waves and projecting them into the liquid medium to generate thrust. Multiple thrusters (transducers) may then provide directional control of the vehicle without requiring movement of the transducer.

[0158] Acoustic waves may be used to apply force (thrust). In general, as the frequency of a sound field increases, the sonic radiation becomes more focused. Focused, high-frequency sound is typically referred to as ultrasound. A sound field that is sufficiently focused applies force on the medium into which it is projected and this force can be proportional to the power of the radiation source. By generating the acoustic streaming from (or through) the walls of the sensing capsule into the fluid medium, the sensing capsule may apply thrust to navigate within the fluid medium.

[0159] Any appropriate source of sonic vibrations may be used. For example, a transducer may be fabricated using piezoelectric materials. Piezoelectric materials can be

actuated by applying a sinusoidal voltage to the front and back of the material, as illustrated in FIG. 30. In this example, the backing and matching layers are layered onto a peizoceramic material to which a voltage is applied, resulting in thrust being applied into the fluid medium. Any suitable piezoelectric material may be used. One example of a material that may be used is APC841 (supplied by APC International), which is a ceramic piezoelectric material. Other piezoelectric materials include quartz, many other ceramic materials (produced by APC International and other companies), and other non-ceramic piezoelectric materials such as polyvinylidene fluoride (PVDF).

[0160] An important characteristic of the piezoelectric material is its acoustic impedance.

The thickness of a piezoelectric material used in a transducer may be dependent on the backing material. If the backing material is rigid (i.e., if its acoustic impedance is much greater than that of the piezoelectric material), at the desired frequency of use, the thickness should be one- quarter wavelength, where the wavelength is defined as the acoustic velocity in the piezoelectric material divided by the frequency of use. If the backing material is non-rigid (i.e., its acoustic impedance is very close to zero), at the desired frequency of use, the thickness should be one- half wavelength. The area of the transducer face can be adjusted based on geometric constraints. The size of the transducer can also be related to the frequency applied (e.g., the smaller the transducer surface area, the higher frequency used to emit a desired thrust), the efficiency of the energy transfer (e.g., the efficiency of transfer of vibration energy into the fluid medium), etc.

[0161] One or more appropriate backing and matching layer materials must be chosen to maximize the transfer of acoustic energy from the transducer to the desired direction of thrust. The backing material is most efficient when it has a low acoustic impedance, like air. An appropriate matching layer or layers can be selected based on the properties of the piezoelectric material and the fluid into which the acoustic energy is transmitted. The thickness of each matching layer should generally be one-quarter wavelength, where the wavelength is defined as the acoustic velocity in the matching material divided by the frequency of use. For a single matching layer, the scientific literature shows that a matching layer of acoustic impedance, where 7 is the impedance of the piezoelectric material, and ^ is the impedance of the fluid, will give the best energy transfer. Thus, one example of an appropriate configuration for generating thrust is a 10 mm diameter APC 841 material that is resonant at 5 MHz with an air backing and an Araldite 502/956 epoxy matching layer 5 mils thick bonded to its surface. In

general, the acoustic stream generator can have a transducer surface diameter of less than 20 mm, less than 10 mm, less than 8 mm, less than 5 mm, less than 2 mm, or less than 1 mm.

[0162] The amount of power that can be projected into the water is theoretically limited by the dielectric breakdown voltage limit of the piezoelectric material, which varies from material to material. In practice, the amount of power is limited by constraints in size and construction to the system that powers the transducer.

[0163] Any appropriate number of acoustic stream generators may be used in a sensing capsule. For example, a sensing capsule may comprise a single acoustic stream generator, a pair of acoustic stream generators, three, four, five, six, or more acoustic generators. These thrusters may be positioned in any appropriate manner over the sensing capsule to allow movement and guidance of the capsule within the fluid medium. In some variations, the sensing capsule comprises an array of acoustic stream generators arranged over at least a portion of the sensing capsule; by selectively activating groups or individual acoustic stream generators, the sensing capsule may be steered or controllably positioned within the fluid medium. For example, four acoustic stream generators may be positioned over the rear of the sensing capsule (e.g., a thruster at the very back, and three radially, equally- spaced thrusters further towards the front of the capsule to turn the capsule). In some variations, the sensing capsule includes acoustic stream generators located primarily at one end of the sensing capsule. Fig. 1 illustrates one variation of a cross-section of a sensing capsule (the "microjects" 50 correspond to the acoustic stream generators). Thruster of different sizes (and powers) may also be used in different combinations. For example, one or more "power" thrusters may be provided to propel the sensing capsule in the fluid medium, and less powerful "manipulation" thrusters may be used to turn or steer the capsule.

[0164] The power supply for the sensing capsule may be appropriately adapted to provide appropriate energy (e.g., sinusoidal power) to activate or control each acoustic stream generator. Thus, the power supply may include logic and/or circuitry to condition the power applied to an acoustic stream generator. In some variations, the frequency and/or amplitude of the power from the power source may be modulated and/or controlled (e.g., from user input, or automatically). Thus, in some variations, it may be useful to control the frequency of power supplied to activate an acoustic stream generator, and thereby control the thrust from the acoustic stream generator.

[0165] A propulsion or guidance control system may be used to navigate the sensing capsule within the fluid medium. The propulsion control system may coordinate the propulsion, the steering (e.g., by selectively activating different transducers), and the buoyancy. In some variations, the capsule may operate as a miniaturize submarine in the fluid medium. The guidance and propulsion control system may comprise guidance control logic to coordinate the activation of the transducers on the sensing capsule to achieve a desired movement. In some variations, the guidance control logic also determines the buoyancy of the capsule. A user may choose the direction of movement and input a command (e.g., move up, move left, move right, move down, etc.) which is then executed by the guidance control logic. Any appropriate user input means may be used, including a joystick, mouse, keyboard, voice command, foot petal, steering wheel, etc.

[0166] In some variations, a buoyancy system is included as part of the sensing capsule or a system including the sensing capsule. The buoyancy system may include one or more chambers for filling or emptying with material to alter the overall buoyancy of the capsule. For example, a buoyancy chamber may be filled by fluid from the fluid medium. Thus, the contents of the buoyancy chamber may be regulated by controlling the evacuating and filling of the microchambers. In one variation, the ballast chamber includes a small piston which moves to evacuate or fill the ballast chamber. In some variations, the ballast chamber may be filled by electrolysis of material (e.g., water).

[0167] The guidance control logic may comprise software, hardware, or a combination of software and hardware, and may be located either within the capsule or external to the capsule (e.g., within a control system which communicates with the capsule). For example, the guidance control logic may comprise software running on a computer which instructs control elements within the sensing capsule by telemetry. The guidance control logic may also include procedures for steady-state operation of the sensing capsule. For example, the guidance control logic may keep the capsule within a relatively constant position within the fluid medium. In some variations, the guidance control logic may process images from the sensor (e.g., stereoimages) and determines position (or change in position) based on the images. This positional information may then be used to correct for movement in position, allowing the maintenance of the capsule position within the fluid medium. In some variations the fluid

capsule position may also be determined based on triangulation from external sensors which detect the location of the capsule.

[0168] For example, an external location system may be used as part of the guidance system, or simply to locate the position of the sensing capsule within a body (including a human body). In some variations, the sensing capsule may emit a signal that can be received (e.g., by externally located receivers positioned around the subject) and used to determine the position of the sensing capsule. In some variations, the sensing capsule includes a marker (e.g., a radioopaque marker, etc.) that can identified by external detectors to determine the position of the sensing capsule. Examples of such identifying systems are well known in the art (e.g., GPS tracking systems, etc.). External tracking or location systems may also be used in conjunction with mapping systems (e.g., systems which determine the location of the sensing capsule relative to structures in the fluid medium sensed by the sensing capsule) and/or with existing maps of the fluid medium into which the sensing capsule is operating.

[0169] Although the examples above describe primarily acoustic non-contact propulsion systems, other non-contact propulsion systems may be used. For example, the sensing capsule may be moved by magnetic fields acting on all or a portion of the sensing capsule or the liquid medium, or by non-acoustic vibrational energy. For example, an externally applied magnetic field may be used to control the motion of the sensing capsule.

Telemetry

[0170] A telemetry system may be used to allow the capsule to communicate with external control or data collection and/or display devices, as indicated in FIG. 2. The telemetry system may include a transmitter and receiver inside of the sensing capsule (or a combined transducer), and a transmitter and receiver attached to the display and/or control device (e.g., a computer). Any appropriate device may be used for the display and/or control device, including a dedicated device having software, hardware or a combination of software and hardware. In some variations, a computer running software is used.

[0171] As described above, one or more additional telemetry devices (e.g., a transceiver relay) may also be used. For example, a transceiver belt (as shown in FIG. 2) may be used to help relay signals to and from the sensing capsule to the display and/or control device.

Commands from the control device may be processed and sent to the sensing capsule via the telemetry system. The sensing capsule may include at least some on-board control logic for interpreting and executing commands from the control device.

[0172] As described above, the telemetry system may also include one or more components for encoding, encrypting or otherwise preparing signals to and from the sensing capsule. Thus, the telemetry system may include a multiplexer for combining signals from different sensors or other signal sources. The telemetry system may also include hardware, software, or a combination of hardware and software for coordinating the sending and receiving of signals. It should be understood that the telemetry system may include a single transducer (or a single transmitter and receiver) for sending all of the sensor information from the sensing capsule. For example, the telemetry system may transmit sensor information from one or more cameras (e.g., both image streams from a pair of stereo cameras) as well as sensor data from other sensors (e.g., pH sensors, temperature sensors, etc.), as described above. These signals may be multiplexed in any appropriate manner.

[0173] Although the capsules described herein are mostly exemplified by use within the gastrointestinal tract, capsules may be used in any appropriate environment. Appropriate environments may be any fluid-filled environment. For example, the sensing capsules described herein may be used to perform endoscopy in the colon for both diagnosis and therapy. As described above, the sensing capsules may be used as a remote sensor for recording (e.g., in real time) measurement of temperature, pH, erythema, inflammation, tumor/mass, etc. via wireless control and communication. These sensing capsules are typically moveable/steerable devices. Furthermore, the sensing capsules may be used with (or without) stereo imaging. For example, a single video or camera may be used.

[0174] Although specific embodiments of the present invention have been illustrated in the accompanying drawings and described in the foregoing detailed description, it will be understood that the invention is not limited to the particular embodiments described herein, but is capable of numerous rearrangements, modifications, and substitutions without departing from the scope of the invention.