Augmented microscopy to improve medical diagnosis

Yaneck Gottesman and his team at Telecom SudParis SAMOVAR laboratory have developed a platform to take on the challenge of combining optics and artificial intelligence to improve the diagnosis of blood disorders. This long-term project, TAMIS, is based on collaborative research between the engineering school, the company TRIBVN and three AP-HP (Paris Hospitals) units (Pitié Salpêtrière, Saint Antoine and Avicenne). It aims to develop a new generation of medical applications.

How did this project come about?


“After completing a thesis on photonics components, I became a professor and researcher in photonics at Telecom SudParis, which has been awarded a Carnot label for its research as part of Carnot Télécom & Société Numérique,” Yaneck Gottesman explains. “This is how I came to participate in a research project conducted in partnership with Morpho, a leader in the field of biometrics.

One of the challenges involved in reinforcing the security of biometric identification systems was to find a way to prevent identity theft which could occur by placing a falsified copy of a fingerprint on a fingertip. We therefore addressed this issue by developing a holographic technique that can perform an in-depth examination of the tissue presented to the reader and obtain a 3D digital representation which can be handled and analyzed to extract discriminant physical measurements.”

"This has become the common theme in my research: how to produce data based on a measurement in order to feed physical models for a better understanding of the tissue of the object being analyzed.  After being contacted five years ago by TRIBVN on the topic of medical diagnosis, we were led to adapt the holography to microscopy in order to analyze biological tissue, and blood smears in particular."


The TAMIS project created with TRIBVN and AP-HP is one of ten winning projects to have been selected for the BPI Health Data Hub call for projects, which promotes and finances e-health projects committed to transparency and the sharing of medical data.

Innovations in imaging and data to be utilized

Plenimage is a Telecom SudParis platform which benefits from the technology mentioned above in two different ways, with quantitative and multimodal data. In fact, two morphological modalities, the tissue absorption coefficient and the optical path, the combination of the local thickness and the optical index, are acquired at the same time. In biology, cells under the microscope are represented by information in the three dimensions with the intensity/phase data couple.

In addition, thanks to synthetic aperture microscopy, we can obtain an extensive resolution measurement.  Synthetic aperture microscopy or SAM is the equivalent of what occurs in astronomical imaging with the Very Large Telescope (VLT). Synthetic aperture involves gathering all the information from various telescopes to reach an apparent diameter equivalent to the greatest distance separating the telescopes. This allows us to see more details. In other words, it increases the resolution of the measurement, which is usually limited to the diameter of the telescope. In microscopy, similar techniques allow us to exceed the theoretical resolution limits with calculations.

The principle involves subjecting a sample to several different conditions of optical excitation and recording the result of the interaction between the light and the object. These different pieces of information are combined by calculations to deduce the results which would be obtained by a microscope with a much larger numerical aperture. This provides us with a comprehensive representation of the cell volume and significantly improved resolution, both in terms of the phase component and intensity component, which provides a morphological phenotype that is much more differentiating than one obtained by traditional optics.

The medical applications

In the case of malaria, infected cells are few in number during the early stages and it is important to be able to identify and characterize them.  The measurements collected using two correlated modalities for the same cell (such as local absorption and the local optical path) can distinguish parasitized red blood cells from healthy red blood cells much more easily, which therefore reduces the rate of false positives or false negatives.

The high resolution should allow for a complete diagnosis of malaria with only a single blood smear, whereas current techniques require several preparation techniques and qualified personnel, which is a drawback in terms of cost and scalability. In fact, WHO fears a resurgence of the parasite, and it will be necessary to monitor the reservoirs in humans and mosquitoes with the least intensive techniques possible. Chemical methods pose limiting logistical challenges (supply, storage, conditions of use, availability of the necessary expertise, etc.).

The TAMIS project focuses on needs in the area of cytohematology for diagnosing and monitoring leukemia patients and those at risk for the disease, by offering effective characterization of the cell lines: leukocytes, red blood cells, and blood platelets.

Further perspectives for the project

Preparation for full-scale experiments is now underway based on the results obtained.

This will involve demonstrating the capacities of this technology in comparison to the reference method. The first phase will involve measuring the detection threshold value for the technique by conducting limiting dilution experiments on blood cultures or blood from parasitized patients. During the second phase, the method’s performance will be assessed under real conditions of use: the automated diagnosis of malaria using only a thin blood smear versus the manual reference method performed by a specialist in two steps, the thick blood smear followed by a thin blood smear.

The catalogue of images being produced by TAMIS will feed the Health Data Hub. It will be available for use by the scientific community for research aimed at creating innovative algorithmic approaches to cell detection and classification based on data of better quality.

The company TRIBVN is the industrial partner for TAMIS. It provides in-depth knowledge on users’ needs and expertise in developing and integrating technical solutions in response to these needs. The imaging needs include the management, sharing, visualization and analysis of data, supported by artificial intelligence. The service provided is the simplification of diagnostic tasks. These solutions are currently produced and marketed on the global level by its subsidiary TRIBVN HealthCare to pathologists at clinics, for research, and for Clinical Research Organizations (CRO) in the field of "Digital Pathology".

The work in microscopy jointly initialized with Telecom SudParis will eventually continue in the framework of a new structure, TRIBVN-life. The purpose of this new structure will be to develop applications in cytohematology and parasitology in order to produce a new generation of “digital reagents” obtained using light/matter interaction probes. These natively digital and multimodal markers (intensity, phase, multi-z and multi-scale) will be even more efficient in feeding artificial intelligence tools for considerable simplification of diagnostic tasks treatment monitoring tasks. Their “scalability” will facilitate their use on a very large scale. At the European level, these innovative markers, developed in Europe, will make the research less dependent on outside contributions, while also limiting the use of chemistry and the transport of biological specimens for diagnosis.

The prospects

Plenimage mobilizes imaging innovations in measurement and processing techniques. The goal is to facilitate the work of diagnostic specialists by performing automatic screenings and simplifying the analysis process using AI. In order to produce a diagnostic report, the specialist will be able to use a digital “optical twin” version for a comprehensive representation of the blood tissue. This will avoid the need to go back to the biological specimen and will simplify the diagnostic tasks and equipment. A saved version will then be able to be used in epidemiology, public health, or for basic research. The vision is to improve the diagnostic accuracy in order to offer patients a better care pathway. The scalability and network connections will facilitate global epidemiological surveillance for cyto-oncology and infectious diseases in the future.