Automated Image Analysis Using Artificial Intelligence
Recent developments at the intersection of deep learning and toxicological image analysis herald a new era of efficiency and accuracy in high-throughput chemical screening. At Sciome, we are working with our clients to exploit the transformative potential of deep learning to solve important problems in this domain. Below we highlight a few recent projects which showcase Sciome’s commitment to addressing the challenges inherent in the analysis of large volumes of image data to provide efficient, standardized, and objective solutions. The results of this work serve not only to automate and enhance the accuracy of image-based assessments but also pave the way for innovative approaches to experimental design and benchmark dose calculations.
Sciome’s foray into deep learning for toxicological image analysis is indicative of a larger paradigm shift happening in scientific research. As the scientific community looks towards more advanced and streamlined methodologies, Sciome stands at the forefront, helping to steer the course towards a future where deep learning and AI transforms toxicological assessments and accelerates the pace of scientific discovery.
Automated Classification of Cell Morphology Changes
Sciome’s platform leverages Convolutional Neural Networks to automate high-throughput chemical screening, achieving over 98% accuracy in distinguishing healthy and altered cell states. This approach streamlines image analysis, reducing manual effort and improving classification precision.
Sciome has developed a platform that addresses the inherent challenge of using advanced imaging platforms to perform high-throughput chemical screening. Implementing this approach would normally require human experts to repeatedly analyze thousands of images of cell cultures treated with various chemicals, making this approach a costly and time-consuming procedure. By employing CNNs, our platform automates the classification of digital assay images, alleviating the subjectivity and time constraints of manual processes. With more than 98% accuracy, the resulting binary classifier can effectively distinguish between healthy and altered cell states for important model systems including differentiated and proliferated HepaRG cells. The multi-class classifier further refines this assessment, assigning granular labels with over 95% accuracy.
Figure 1: Example input image, and the output generated by the model, with prediction scores
This work has been recently published in a study that meticulously explores various CNN architectures, showcasing Sciome’s commitment to optimizing performance. The utilization of Class Activation Maps (CAM) adds a layer of transparency, allowing for a deeper understanding of the neural network’s inner workings and leading to additional algorithmic refinements. The results underscore the platform’s capability to deliver highly accurate and reproducible cytotoxicity assessments across diverse cell types, setting a benchmark for automated image classification in chemical screening.
Accelerating Benchmark Dose Calculation through Deep Learning
Sciome is also making important efforts to address the resource-intensive nature of calculating benchmark dose values, which are traditionally reliant on conventional methods and apical endpoints or transcriptomic datasets. We have introduced a method that employs deep learning to classify assay images based on cell health, ultimately achieving over 98% accuracy. After an appropriate statistical transformation, the resulting model predictions can be used as in silico endpoints for the purpose of Benchmark Dose (BMD) modeling. Remarkably, the correlation between image-derived BMD values and those calculated using transcriptomic datasets surpasses 90%, suggesting the viability of high-throughput imaging as a faster and more efficient alternative.
Figure 2: Dose response curves for CPZ and Menadione. The binary model’s class prediction probability is plotted against the dose concentration for the treated chemicals in each of the replicates
Pathway enrichment analysis of correlated transcriptomic responses further sheds light on potential applications, emphasizing the broad implications of Sciome’s deep learning approach in guiding experimental design. The results of this recent work are highlighted in a recent publication.
Publications and Presentations
Tandon, A., Howard, B., Mav, D., Balik-Meisner, M., Ramaiahgari, S., Ferguson, S., Shah, R., Merrick, A. (2023).” Deriving Benchmark Dose from the Deep Learning Prediction Scores of High-Throughput Toxicology Images”. Podium presentation at Society of Toxicology 2023 Nashville, TN.
Tandon A, Howard B, Ramaiahgari S, Maharana A, Ferguson S, Shah R, Merrick A (2022). “Deep Learning Image Analysis of High-Throughput Toxicology Assay Images”. SLAS Discovery 1:9; 27-38.
Tandon, A., Howard, B., Ramaiahgari, S., Maharana, A., Ferguson, S., Shah, R., Merrick, A. (2022). “Deep Learning Image Analysis of High-Throughput Toxicology Assay Images”. Poster at Society of Toxicology 2022 San Diego, CA. Selected as 2022 CTS Top 10 poster.
Tandon, A., Maharana, A., Howard, B., Ramaiahgari, S., Dunlap, P., Rice, J., Merrick, A., DeVito, M., Ferguson, S., Shah, R. (2020). “Towards Automation of High-Throughput Toxicology Assay Image Analysis Using Deep Learning”. Oral presentation at ASCCT 9th Annual Meeting.
Maharana, A., Howard, B.E., Tandon, A., Ramaiahgari, S., Dunlap, P., Rice, J., Merrick, A., Devito, M., Ferguson, S., Shah, R. (2019). “Deep Learning Image Analysis of High-Throughput Toxicology Assay Images”. Poster at Society of Toxicology 2019 Baltimore, MD
3D Spheroid Analysis
The use of 3D spheroids in toxicology research enhances the study of cellular behavior and drug responses by replicating them using in vivo-like environments. To improve analysis, Sciome developed an AI-based approach that integrates image processing and machine learning to detect, segment, and analyze spheroid structures. This innovation enables more precise characterization of chemical effects on morphology and viability.
The use of 3D spheroids in toxicology research has gained significant attention due to their ability to replicate the intricate architecture and dynamic environment of tissues more accurately than traditional 2D cell cultures. This allows for a better understanding of cellular behavior, drug responses, and toxicity, as 3D spheroids mimic the complex cell-cell and cell-extracellular matrix (ECM) interactions that occur in vivo. In response to the need for a more precise and automated method to analyze 3D spheroid images, Sciome has developed an AI-based approach that integrates advanced digital image processing and machine learning techniques to detect, segment, and analyze cells and nuclei within spheroids. This innovation enables a more detailed and accurate characterization of how chemicals affect spheroid morphology and viability.Below is an expanded description of the key aspects of Sciome’s method:
1. Cell and Nucleus Detection and Segmentation
Sciome’s AI-powered image analysis pipeline can efficiently identify the boundaries of both cells and nuclei within 3D spheroid images. This is performed by:
Figure 3b: Nuclei prediction in liver spheroid by the segmentation modelFigure 3a: Cells prediction in liver spheroid by the segmentation modelCell Boundary Identification: Using deep learning models and advanced image processing techniques to accurately delineate the outer contours of each cell within a spheroid, even in densely packed structures.
Nucleus Identification: Detecting and segmenting individual cell nuclei based on staining, intensity, and structural information across different image channels (e.g., DAPI, Hoechst staining for nuclei).
2. Cell and Nucleus Counting
After detection and segmentation, the algorithm can count the number of individual cells and nuclei present in a spheroid. This is crucial for:
Quantifying Proliferation: A higher cell count could indicate increased proliferation rates, while a lower count could suggest apoptosis or cytostatic effects due to chemical exposure.
Figure 4: Nuclei prediction in kidney spheroid
Assessing Viability: The ratio of nucleus count to cell count helps in determining whether the cells have intact nuclear structures, providing insights into cell health and death.
3. Morphological Feature Quantification
Sciome’s method goes beyond simple counting by extracting a wide range of morphological features from the segmented cells and nuclei:
Area (2D Slice): The cross-sectional area of each cell and nucleus from 2D image slices is measured, helping to evaluate size variations across the sample.
Volume (3D): Using 3D reconstruction techniques, the total volume of individual cells and nuclei is calculated. This is particularly important for analyzing how certain chemicals might cause cells to swell, shrink, or deform.
Shape Metrics: Various shape descriptors, such as circularity, aspect ratio, elongation, and surface roughness, are computed. Changes in these parameters can reflect chemical-induced alterations in cellular morphology, such as stress responses or necrosis.
4. Spheroid Classification and Benchmark Dose (BMD) Calculation
An additional layer of analysis classifies the spheroid as intact or disrupted based on its structural integrity. This classification is important for:
Determining Spheroid Health: Spheroids that maintain their structure are considered intact, whereas disruptions in cellular organization may indicate toxic effects.
BMD Calculation: The method calculates the Benchmark Dose (BMD) for chemicals, which is the dose at which a specified degree of effect (e.g., 10% disruption in spheroid integrity) is observed. This is crucial for determining chemical safety and toxicity thresholds.
Figure 5: Maximum Intensity Projection (MIP) images of spheroids exposed with different doses of a chemical. For higher doses, the cells start to disintegrate which results in the unstable spheroid shape.
5.Dynamic Spheroid Tracking and Time-Lapse Analysis
Sciome’s method is also capable of tracking spheroid behavior over time, which is particularly useful for long-term studies:
Spheroid Tracking in Widefield Images: By capturing sequential widefield images, the system can track changes in spheroid position, size, and structure over time. This enables researchers to monitor the growth or shrinkage of spheroids under different chemical treatments.
Morphological Changes Over Time: The method evaluates temporal changes in key morphological features such as size, shape, and compactness. This is critical for understanding how the exposure to chemicals affects spheroid dynamics, such as growth rate, compaction, or degradation over time.
6. Applications in Toxicology
The combination of high-throughput image analysis, morphological feature extraction, and dose-response assessment positions Sciome’s method as a valuable tool in toxicology for:
Drug Screening: Identifying the effects of potential therapeutics or toxicants on spheroid morphology to assess drug efficacy or toxicity.
Chemical Risk Assessment: By calculating the BMD and quantifying morphological disruption, the tool can provide actionable insights for chemical risk assessment and regulatory decision-making.
Mechanistic Studies: Sciome’s method allows for a deeper investigation into the mechanisms of action of various chemicals by correlating observed morphological changes with specific biological pathways, such as apoptosis, stress responses, or cell proliferation.
Multi-omics Analysis
Integrating 3D spheroid imaging with multi-omics approaches, especially transcriptomics, enhances our understanding of how chemical treatments affect both cellular morphology and molecular pathways. This multi-omics approach bridges the gap between observable phenotypic changes and underlying molecular alterations, offering a holistic view of cellular responses to chemical exposures. Sciome’s advanced AI-based imaging analysis, combined with transcriptomic profiling, provides a powerful framework for this integrated analysis.
To achieve comprehensive insights, the morphological data from imaging analysis is integrated with corresponding transcriptomic data using advanced statistical and machine learning models. This integration is crucial to establish correlations between structural changes and molecular mechanisms. The following are some of the key analyses:
Feature Correlation Analysis: Identifying significant correlations between specific morphological features (e.g., increased cell size, disrupted spheroid integrity) and changes in gene expression (e.g., up regulation of cell cycle-related genes, downregulation of apoptosis regulators) using Canonical Correlation Analysis (CCA).
Multivariate Regression Models: Multivariate regression can be used to predict gene expression changes from multiple morphological features or vice versa. These models help establish a predictive framework where observed changes in spheroid morphology can be linked to specific gene expression patterns.
Pathway Mapping: Changes in morphology are mapped to pathways, allowing researchers to determine which signaling pathways are likely responsible for the observed morphological effects.
Sciome’s AI-based approach to 3D spheroid image analysis offers an advanced, comprehensive tool for studying the effects of chemicals on cellular morphology. Through automated detection, segmentation, and tracking, integrated with expert transcriptomic analysis, Sciome’s methods enable a holistic approach to 3D spheroid models, linking morphological changes to molecular mechanisms. This comprehensive analysis enhances our understanding of how chemicals affect cellular behavior and structure, providing valuable insights for drug development, chemical risk assessment, and the study of disease mechanisms.
Presentations
Oktay A, Tandon A, Howard B, Phadke D, Mav D, Balik-Meisner M, Pearson A, Ferguson S, Shah R. “Evaluating Botanical Safety Using AI -Based Quantification of 3D Liver Spheroids with Cell Painting Assay and High Throughput Transcriptomics”. Poster at Society of Toxicology, Orlando, FL (2025)
Oktay A, Tandon A, Howard B, Phadke D, Mav D, Balik-Meisner M, Pearson A, Ferguson S, Shah R. “Evaluating Botanical Safety Using AI -Based Quantification of 3D Liver Spheroids with Cell Painting Assay and High Throughput Transcriptomics”. Poster at American Society for Cellular and Computational Toxicology (2024)
Zebrafish are a key model in biomedical research, widely used for high-throughput drug and gene screening. To address the challenges of visually assessing larval morphological changes, we developed a multi-view convolutional neural network (MVCNN). By leveraging both dorsal and lateral embryo views, our model achieved higher accuracy than single-view CNNs, enabling automated classification of developmental malformations. This approach streamlines assessments, ensuring more objective and standardized evaluations.
Figure 6: GRAD CAM of model predictions of different morphological changes
Zebrafish are widely used as a model organism in biomedical research, especially for high-throughput screening of drugs and genes. Interest in zebrafish as a model organism has grown at an annual rate of 13% over the past 15 years, as measured by peer-reviewed publications. Routinely used to study various biological processes and human diseases, zebrafish have contributed to significant medical and scientific breakthroughs, such as gene therapy, organ regeneration, and drug discovery. We worked with our clients to tackle the challenges of visually assessing larval morphological changes in zebrafish embryos—traditionally, a resource-intensive and subjective process. By developing a customized multi-view convolutional neural network, we leveraged the distinct dorsal and lateral views of each embryo to achieve higher accuracy than conventional single-view CNN models. This enables automatic classification of several common developmental malformations with high accuracy, expediting the assessment process and ensuring more objective, standardized outputs.
Our MVCCN classification models showed high accuracy for 13 out of 20 common morphological changes, with an F1 score greater than 0.80. These models allow researchers to quickly screen zebrafish larva images exposed to different chemicals, leaving the more ambiguous or unusual images for expert screeners. Our segmentation models also demonstrated high accuracy for most regions, allowing users to quantify the effects of chemical exposure more accurately and perform detailed downstream analyses of morphological changes.
Figure 7: Example of Segmentation model’s prediction
Together, these automated models save time while producing results that are more accurate, reproducible, and less biased than manual assessments. They represent a cutting-edge New Approach Methodology (NAM) in zebrafish image analysis and highlight Sciome’s commitment to leveraging deep learning for programmatic decisions in chemical evaluation, offering a paradigm shift in efficiency and reliability. The results of this work are highlighted in several recent publications.
Table 1: MVCNN Vs Single View CNN performance metrics comparison table
Table 2: Segmentation model performance metrics table
Publications and Presentations
Tandon A, Howard B, Green A, Merrick A, Shah R, Shockley K, Cunny H, Ryan K, Hsieh H. Artificial Intelligence (AI)-Driven Morphological Assessment of Zebrafish Embryos for Developmental Toxicity Chemical Screening. Submitted to Aquatic Toxicology. (2025)
Tandon A, Howard B, Green A, Merrick A, Shah R, Shockley K, Cunny H, Ryan K, Hsieh H. Artificial intelligence (AI)-Driven Morphological Assessment of Zebrafish Embryo for Developmental Toxicity Chemical Screening. Poster at Society of Toxicology’s 63rd Annual Meeting and ToxExpo. (2024)
