Fraunhofer Institute for Integrated Circuits IIS
Fraunhofer Institute for Integrated Circuits IIS

Digital Pathology

MIA – Microscopy Image Analysis

MIA – Microscopy Image Analysis
© Fraunhofer IIS

We integrate our image analysis "apps" into our MIA software platform. MIA consists of

  • Virtual Microscopy Viewer
    • Designed for high throughput creation
    • Developed for pyramidal (gigapixel) brightfield and fluorescent whole-slide-images
    • Support for z-Layers (allows for interactive refocusing)
    • Synchronized grid view of up to 4x4 slides
    • Support of multiple scan areas per slide
    • Powerful annotation system
    • skinnable and customizable UI
  • App-Center for image analysis apps 
    • Results can be visualized live
    • Brightfield and fluorescence
    • Analysis of either ROI, Field of View or Whole Slide
    • Batch-analysis of multiple slides
    • Accommodates both AI or “classical” machine vision
  • Extensible WSI format
    • Support for GPU-accelerated encoding
    • JPEG2000, JPEG or other compression formats
    • “Sparsely” scanned slides do not have to be “filled” to a rectangular area
    • Tiles can be arbitrarily positioned and overlap each other – no predefined grid layout

We offer C++ or .NET language bindings for our MIA-platform. The MIA viewer can be licensed as a re-branded turn-key ready-to-install solution.

MIA Virtual Microscopy Viewer

Detection of Colorectal Carcinoma and Tissue Partitioning

Goal: partition different tissue types and detect adeno carcinoma in H&E stained colon tissue sections

Partners: joint effort with the Institute of Pathology of the University Hospital Erlangen

Current Status:

  • Created annotated database of 20x H&E slides
  • Currently training AI models to classify patches from Whole-Slide
  • Currently rescanning slides with other scanners to become more robust to scanner variations

Database:

  • over 300 whole-slides that amount to nearly 30 mio. tiles of size 227x227
  • many classes: tumor, tumor cell clusters, fat, mucosa, connective tissue, blood vessels, lymphoid follicles

Solution:

  • end to end training of Convolutional Neural Network

Privacy warning

With the click on the play button an external video from wwww.youtube.com is loaded and started. Your data is possible transferred and stored to third party. Do not start the video if you disagree. Find more about the youtube privacy statement under the following link: http://www.youtube.com/t/privacy_at_youtube

Colon Histopathology Cartography

PathoScan Project: Automated Digitalization in Routine Pathology

PathoScan: Artificial Intelligence in routine pathology
© ©Vshyukova - stock.adobe.com
PathoScan – Optimization of morphological work processes through an automated digitized pathology system

Goal: Digital and automated microscopy workflow including pre-scan QC using an overview camera

Partner: PreciPoint GmbH, University of Regensburg, Technical University of Munich TUM, HTI bio-X GmbH

             

Current Status:

  • Systematic collection and prioritization of possible error artifacts that can occur during sample preparation
  • Collection and digitization of problematic slides

Database: Slides are currently being collected in Regensburg and Munich and then digitized. Using the database, the scientists at Fraunhofer IIS will develop and validate the image analysis algorithms.

Our Solution:

  • Coloring (HTI Smart Automation) and digitization (PreciPoint) using a fully automated digitization workflow
  • Automatic image analysis using the overview camera to detect various quality problems such as protruding cover glasses, air bubbles, foreign particles, folds, cracks
  • Detection of over / under-staining or faulty staining by objective quantification of the coloring
  • Cost and time efficient quality assurance procedures

Tumor Budding

© Fraunhofer IIS
© Fraunhofer IIS

Tumor budding is an independent predictor of lymph node metastases in pT1 colorectal cancer and predictor of survival in stage II colorectal cancer1.   

Goal: automatic scoring

Partners: joint effort with the Institute of Pathology of the University Hospital Erlangen

Current Status:

  • developed automatic analytics App for an immunohistochemistry staining (pan-cytokeratin – PCK) that is able
  • to detect tumor invasion front
  • to detect tumor buds
  • to detect hot spots and determine low/intermediate/high budding based on cut offs
  • algorithm trained on 40x PCK-stained whole-slides

Database: over 100 whole-slides with over 50.000 annotated tumor buds

Solution:

  1. Detect tumor bud candidates using classical image processing that evaluates size, color, and distance to tumor
  2. Weed out false positives using a Convolutional Neural Network trained to classify tumor buds

------------------------------------------------------------------

1Lugli et al, Recommendations for reporting tumor budding in colorectal cancer based on the International Tumor Budding Consensus Conference (ITBCC) 2016

 

Low-Budget Diagnostics of Gastroesophageal Tumors

DigImmune
© Fraunhofer IIS

Goal: histological image analysis – comparison iSTIX vs whole-slide-scanner

Partners: joint effort with the Definiens AG, Institute of Pathology of the University Hospital Erlangen, Institute of Pathology of Bayreuth Hospital

Current Status:

  • Creation of a comprehensive annotated database: data from various clinical centers, different scanners and staining methods (H&E plus 9x immunohistochemistry), both of resections and of biopsies
  • Quantification of IHC samples and analysis of correlations between IHC biomarkers and between patients

Database: completed

  • A total of 1000 slides (100 cases with 10 stains each) from Bayreuth and Erlangen digitized with 3DHISTECH scanners
  • Additional overview scan (50x) of the H&E staining of all cases manually scanned with iSTIX
  • Additionally, tumor region selected by the pathologist in H&E staining with iSTIX in high resolution (200x) manually scanned

Solution:

  • Scan whole-slides with Scanners
  • Scan tumor regions with iSTIX
  • Image analysis of whole-slides created with iSTIX vs. scanner

 

"DigImmune (Digitale Diagnostik für die Immuntherapie von Krebspatienten)"

Single Cell Tumor Detection in Lymph Nodes

© Fraunhofer IIS

Goal: early detection of cancer metastasis by detecting even single disseminated tumor cells in a lymph node.

Partners: joint effort with the Fraunhofer ITEM and Fraunhofer IPA

Current Status:

  • Development of single components (tissue grinder, scanning and image analysis, single cell DNA amplification, laboratory notebook software) finished
  • Evaluation ongoing

Database:

  • ~50 slides of melanoma and lung cancer

Solution:

  • Prior work by University Tübingen and Fraunhofer ITEM has shown that even a presence of only a handful of disseminated cancer cells per million lymphocytes has a negative impact on survival
  • Fraunhofer IPA has developed TissueGrinder for dissolving a lymph node into a cell suspension
  • Immunocytological staining using melanoma marker (gp100)
  • Scanning of cell suspension with Fraunhofer IIS’s SCube scanning platform
  • Fraunhofer IIS image analysis: detection of tumor cell candidates and tumor vs. artefact classification with Convolutional Neural Network
  • Molecular single cell analysis with assay developed by Fraunhofer ITEM (commercially available as Ampli1TM WGA)

Seeking Commercialization Partners


-----------------------------------------------------------------

Further Reading:
1. Ulmer A., et al., Quantitative measurement of melanoma spread in sentinel lymph nodes and survival. PLoS Med, 2014. 11(2): e1001604.
2. Ulmer A., et al., The sentinel lymph node spread determines quantitatively melanoma seeding to non-sentinel lymph nodes and survival. Eur J Cancer, 2018. 91: p. 1-10
3. Werner-Klein M., et al. Genetic alterations driving metastatic colony formation are acquired outside of the primary tumour in melanoma. Nat Commun, 2018. 9(1): p. 595

xAI – Explainable AI in Digital Pathology

© BMBF

Goal: turn AI algorithms used in Digital Pathology from a black box into a glass box

Partners: University of Bamberg (Cognitive Systems, Prof. Dr. Ute Schmid), Fraunhofer HHI

Current Status:

  • Research and evaluation of technologies to make AI algorithms more explainable

Solution:

  • Display results from neural networks in an easily comprehensible and transparent manner
  • Take into account uncertainties from the annotated training data as well from the model itself

Funded by Federal Ministry of Education and Research (BMBF)
Transparent Medical Expert Companion (TraMeExCo)

---------------------------------------------------------

Literature:
Schmid, Wittenberg (2019) TraMeExCo - Medical Expert Companion (Vortrag). All-Hands-Meeting (5. Juni 2019, Dortmund), Förderschwerpunkt Maschinelles Lernen des BMBF. -- [Slides], [Poster]
Rieger, Finzel, Seuß, Wittenberg, Schmid (2019) Make Pain Estimation Transparent: A Roadmap to Fuse Bayesian Deep Learning and Inductive Logic Programming (Poster). 41st Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC, 23.-27. Juli 2019, Berlin)

Bruns, Volker, Geppert, Carol (2019), Einsatz von Künstlicher Intelligenz in der digitalen Pathologie - Auf dem Sprung in die Routine?, Trillium Diagnostik 2/2019, https://www.trillium.de/zeitschriften/trillium-diagnostik/ausgaben-2019/td-heft-22019/pathologie/einsatz-von-kuenstlicher-intelligenz-in-der-digitalen-pathologie.html

Development of Robust AI systems for Digital Pathology

Goal: design AI algorithms that are robust to real-world heterogeneity (different scanners, staining variances, sub-optimal slide quality)

Current Status:

  • Developed algorithms for simulating staining variances
  • Creating multi-scanner database of H&E stained colon section
  • Developing Prototypical Networks (PN) for Few-Shot Learning: this will enable the development of new classifiers using only little data and few annotations

Database

  • In development

Solution:

  • Represent real-world heterogeneity in training database or normalize data to match training data characteristics

Research Project: ADA Center – Applications Data Analytics
Subproject: Explainable AI in Life Science and Automotive

Events and trade fairs

 

© Messe München

The world's leading trade fair for laboratory technology, analysis, biotechnology will be held purely virtual from October 19 - 23!

 

Save the date - we look forward to get in touch with you!

To the event website
 

© Fraunhofer

Digital Fraunhofer event

Fraunhofer Solution Days 2020

Presentation. Inspiration. Vision.

What can you expect?

On October 26–29, 2020, we will address four topics that are extremely relevant for the power of innovation in Germany and Europe:

Health. The digital economy. Systems and mechanical engineering. Mobility.

 

#WeKnowSolutions
 

Virtuelle Veranstaltung

7th Digital Pathology & AI Congress

Utilizing AI & digital pathology to advance pathology practice & enable enhanced patient care

 

The 7th Digital Pathology & AI Congress will take place virtually from November 11th - 13th.

 

 

Digital Pathology & AI Congress