Mastering Cell Identity: A Comprehensive Guide to Hierarchical Classification with scClassify

Abstract

This article provides a thorough exploration of scClassify, a state-of-the-art tool for hierarchical cell type classification in single-cell RNA sequencing data. Tailored for researchers and drug development professionals, we cover its foundational principles rooted in ensemble learning and cell type hierarchies. The content extends to practical implementation, from installing the R/Bioconductor package and training models to advanced multi-reference analysis. We address common troubleshooting scenarios and optimization techniques, including sample size estimation and handling unassigned cells. Finally, we validate its performance against other methods, highlight its proven accuracy across diverse tissues, and discuss its evolving applications in biomedical research, including its next-generation iteration, scClassify2, for identifying sequential cell states.

What is scClassify? Unpacking the Framework for Hierarchical Cell Typing

The Challenge of Accurate Cell Type Identification in scRNA-seq Data

Single-cell RNA sequencing (scRNA-seq) has revolutionized biomedical research by enabling the profiling of gene expression at the individual cell level, revealing unprecedented insights into cellular heterogeneity within complex tissues and organisms [1] [2]. Since its conceptual breakthrough in 2009, scRNA-seq technology has rapidly evolved, with throughput increasing from a few cells per experiment to hundreds of thousands of cells, while costs have dramatically decreased [1]. This technological advancement has made it possible to classify, characterize, and distinguish individual cells at the transcriptome level, leading to the identification of rare but functionally important cell populations [1].

However, accurate cell type identification remains a significant computational challenge in scRNA-seq data analysis [3]. The traditional approach relies on unsupervised clustering followed by manual annotation based on known marker genes, a process that is inherently subjective, time-consuming, and biased toward better-characterized cell types [3]. With the exponential growth in both the scale and complexity of scRNA-seq datasets, researchers now require sophisticated computational frameworks that can leverage existing annotated references to automate and improve the accuracy of cell type identification while accounting for the hierarchical nature of cell type relationships [3]. This application note explores these challenges and presents a hierarchical classification framework as a robust solution for accurate cell type identification.

Technical Hurdles in scRNA-seq Cell Typing

The journey from tissue sample to cell type identification involves multiple technical steps where challenges can arise, potentially compromising the accuracy of final results. Understanding these hurdles is essential for developing effective solutions and interpreting data correctly.

A primary concern begins at the sample preparation stage, where the dissociation of tissues into single-cell suspensions can induce artificial transcriptional stress responses [1]. Studies have confirmed that protease dissociation at 37°C can artificially alter cellular transcriptomes, leading to inaccurate cell type identification [1]. Dissociation at 4°C or utilizing single-nucleus RNA sequencing (snRNA-seq) instead of whole-cell approaches has been suggested to minimize these artifacts, particularly for sensitive tissues like brain, muscle, and various tumor tissues [1].

The table below summarizes major technical challenges in scRNA-seq workflows and their impact on cell type identification:

Table 1: Key Technical Challenges in scRNA-seq Cell Type Identification

Challenge Category	Specific Issues	Impact on Cell Typing
Sample Preparation	Artificial stress responses during tissue dissociation [1]	Altered transcriptional patterns mimic different cell states
RNA Capture & Amplification	Low mRNA amounts, inefficient capture, amplification biases [2]	"Dropout" events where genes are not detected, limiting marker gene identification
Sequencing Artifacts	Ambient RNA contamination, doublets (multiple cells labeled as one) [4] [5]	False cell types appear due to mixed expression profiles
Data Quality Issues	High noise, sparsity, batch effects between experiments [2] [6]	Reduces ability to distinguish biologically distinct populations
Biological Complexity	Continuous cell states, transitional populations, novel cell types [3]	Hard discrete classifications miss biological reality

Once sequencing data is generated, additional computational challenges emerge. scRNA-seq data are characterized by high dimensionality, technical noise, and sparsity—often described as "dropout" events where transcripts are not detected due to the limited sensitivity of the assay [2] [6]. Batch effects—systematic technical variations between experiments conducted under different conditions or by different personnel—can obscure genuine biological variations and complicate the integration of multiple datasets [6] [5]. Furthermore, ambient RNA contamination, where free-floating transcripts from damaged cells are captured and barcoded alongside intact cells, can create artificial expression profiles that masquerade as distinct cell types [4] [5]. These technical artifacts, if not properly addressed, can lead to misclassification of cell types and erroneous biological interpretations.

Hierarchical Classification with scClassify: A Framework for Accuracy

Conceptual Framework and Algorithmic Approach

The scClassify framework addresses fundamental limitations in traditional cell type identification by adopting a multiscale, hierarchical approach that mirrors the biological reality of cell type relationships [3]. Unlike "one-step" classification methods that directly assign cells to terminal cell types, scClassify first constructs a cell type tree from reference data, organizing cell types in a hierarchy with increasingly fine-grained annotations [3]. This hierarchical organization allows the algorithm to capture the natural relationships between broad cell categories and their specialized subtypes.

The algorithm employs ensemble learning to enhance classification accuracy and robustness [3]. Rather than relying on a single classification model, scClassify combines multiple weighted k-nearest neighbor (kNN) classifiers trained using different gene selection methods (including differential expression genes) and similarity metrics [3]. This ensemble approach captures diverse aspects of cell type characteristics that might be missed by any single method. At each branch node of the cell type hierarchy, these ensemble classifiers make predictions, ultimately integrating results across the entire tree structure to generate final cell type assignments [3].

A particularly innovative feature of scClassify is its sample size estimation capability [3]. The framework can estimate the number of reference cells required to accurately discriminate between cell types at any level in the hierarchy by fitting an inverse power law to pilot data [3]. This functionality provides crucial guidance for experimental design, ensuring that reference datasets contain sufficient cells to support reliable classification, particularly for distinguishing closely related subtypes.

Experimental Protocol for Hierarchical Classification

Implementing scClassify requires careful attention to both experimental design and computational procedures. The following protocol outlines the key steps for applying hierarchical classification to scRNA-seq data:

Sample Preparation and Sequencing

• Cell Isolation: Prepare single-cell suspensions using gentle dissociation protocols, considering snRNA-seq for sensitive tissues [1]. For complex tissues, fluorescence-activated cell sorting (FACS) or microfluidic separation can be employed [2].
• Library Preparation: Select appropriate scRNA-seq protocol based on research goals. For high-throughput studies, 3'-end counting methods (e.g., 10x Genomics, Drop-seq) are suitable; for detailed transcriptome characterization, full-length methods (e.g., Smart-Seq2) are preferable [2].
• Sequencing: Sequence libraries following platform-specific recommendations, ensuring sufficient depth (typically 50,000-100,000 reads per cell) to capture cell type-specific markers [7].

Data Preprocessing and Quality Control

• Raw Data Processing: Process FASTQ files using appropriate pipelines (e.g., Cell Ranger for 10x Genomics data) to generate gene expression matrices [4].
• Quality Control: Filter out low-quality cells using these thresholds [4] [8]:
  • Remove cells with unusually high or low UMI counts (potential multiplets or empty droplets)
  • Exclude cells with high mitochondrial read percentages (>10-20%, indicating dying cells)
  • Eliminate cells with aberrant numbers of detected genes
• Ambient RNA Correction: Apply computational tools like SoupX or CellBender to remove contamination from ambient RNA [4].

Reference Dataset Construction

• Cell Type Annotation: Begin with well-annotated reference datasets, using established marker genes and conservative approaches to define cell populations [3].
• Hierarchy Construction: Build cell type hierarchies using tools like HOPACH, organizing cell types from broad categories to fine subtypes based on transcriptional similarities [3].
• Sample Size Assessment: Use scClassify's estimation functionality to determine if reference datasets contain sufficient cells for robust classification, particularly for rare populations [3].

Classification Implementation

• Classifier Training: Train ensemble classifiers using multiple gene selection methods and similarity metrics at each node of the cell type hierarchy [3].
• Query Dataset Processing: Normalize and scale query datasets using the same parameters applied to reference data [3].
• Hierarchical Classification: Apply scClassify to query data, allowing for assignment to intermediate nodes when sample size is insufficient for confident subclassification [3].
• Validation: Validate classification results using known marker genes and cross-validation approaches [3].

The following diagram illustrates the complete scClassify workflow, from raw data processing to final cell type assignment:

scClassify Hierarchical Workflow

Performance Advantages and Validation

scClassify has demonstrated superior performance compared to other supervised cell type identification methods across diverse datasets and experimental conditions [3]. In comprehensive benchmarking involving 114 pairs of reference and testing datasets representing diverse sizes, technologies, and complexity levels, scClassify consistently achieved higher accuracy rates than alternative methods [3]. The performance advantage was particularly pronounced in challenging scenarios where test datasets contained cell types not present in the reference data—a common situation in real-world research applications.

The ensemble learning approach of scClassify proved particularly valuable, as it frequently achieved classification accuracy higher than that of even the single best individual model [3]. By combining multiple weak classifiers, the ensemble approach captures complementary aspects of cell type characteristics, resulting in more robust and accurate predictions across diverse cell types and experimental conditions. Additionally, scClassify has demonstrated excellent scalability, efficiently handling datasets with large numbers of cells comparable to other existing methods [3].

Essential Research Reagents and Computational Tools

Successful implementation of hierarchical classification requires both wet-lab reagents for sample preparation and computational tools for data analysis. The following table catalogues essential resources for scRNA-seq experiments focused on accurate cell type identification:

Table 2: Essential Research Reagents and Computational Tools for scRNA-seq Cell Typing

Category	Item	Function/Purpose
Wet-Lab Reagents	Tissue dissociation kits (e.g., gentle MACS) [1]	Isolation of viable single cells while minimizing stress responses
	Nuclei isolation buffers (for snRNA-seq) [1]	Alternative to whole-cell isolation for sensitive or frozen tissues
	Viability dyes (e.g., propidium iodide) [8]	Identification and removal of dead cells during preparation
	Barcoded beads (e.g., 10x Genomics Gel Beads) [7]	Cell-specific barcoding during library preparation
	Reverse transcription and cDNA amplification kits [1]	Conversion of limited mRNA to amplifiable cDNA
Computational Tools	scClassify [3]	Hierarchical cell type classification using ensemble learning
	Seurat [9] [8]	Comprehensive R-based toolkit for scRNA-seq analysis
	Scanpy [9]	Python-based scalable single-cell analysis
	Cell Ranger [9] [4]	Processing 10x Genomics data from FASTQ to count matrices
	SoupX/CellBender [9] [4]	Removal of ambient RNA contamination
	Scrublet/DoubletFinder [5]	Identification and removal of doublet cell barcodes
	Harmony [9]	Batch effect correction across multiple datasets

The computational ecosystem for scRNA-seq analysis has expanded dramatically, with tools now available for every stage of the analytical workflow. Foundational platforms like Seurat (for R users) and Scanpy (for Python users) provide comprehensive environments for data preprocessing, normalization, dimensionality reduction, clustering, and visualization [9]. Specialized tools like Harmony effectively correct batch effects between datasets, while CellBender employs deep learning to remove ambient RNA noise [9]. For researchers working with 10x Genomics data, the Cell Ranger pipeline provides a standardized approach for processing raw sequencing data into gene expression matrices [4].

The following diagram illustrates the hierarchical structure of cell type classification, showing how broad cell categories branch into increasingly specific subtypes:

Cell Type Hierarchical Tree

Applications in Biomedical Research and Drug Development

The hierarchical classification approach for scRNA-seq data has transformative potential across multiple domains of biomedical research and therapeutic development. In cancer research, precise identification of tumor subpopulations, immune infiltrates, and stromal components within the tumor microenvironment provides insights into disease mechanisms and potential therapeutic vulnerabilities [8] [6]. The ability to distinguish rare cell populations, such as cancer stem cells or drug-resistant clones, enables the development of more targeted treatment strategies.

In drug discovery and development, hierarchical classification facilitates the identification of specific cell types affected by therapeutic interventions and helps elucidate mechanisms of action and toxicity [2]. When applied to patient-derived organoids or animal models treated with candidate compounds, this approach can determine cell type-specific responses and identify biomarkers of efficacy or toxicity [8]. Furthermore, by accurately classifying immune cell subsets in clinical samples, researchers can better understand and predict immunotherapeutic responses and immune-related adverse events.

The framework also advances personalized medicine approaches by enabling precise characterization of patient-specific cellular heterogeneity [2]. In complex diseases, different patients may exhibit distinct cellular subpopulations driving pathology, requiring tailored therapeutic approaches. Hierarchical classification helps identify these patient-specific patterns, potentially guiding treatment selection and biomarker development for targeted therapies.

The field of single-cell genomics continues to evolve rapidly, with several emerging trends likely to shape the future of cell type identification. Multi-omic integration—combining scRNA-seq with measurements of chromatin accessibility (scATAC-seq), protein expression, spatial context, and other modalities—will provide increasingly comprehensive views of cellular identity and function [6]. Hierarchical classification frameworks like scClassify are well-positioned to incorporate these additional data layers, further improving classification accuracy and biological relevance.

Advancements in artificial intelligence and deep learning are beginning to transform cell type identification, with tools like scvi-tools bringing deep generative modeling into the mainstream of single-cell analysis [9]. These approaches can model the noise and latent structure of single-cell data, providing superior batch correction, imputation, and annotation compared to conventional methods [9]. As these technologies mature, they may be incorporated into hierarchical frameworks to enhance their performance and capabilities.

The creation of comprehensive cell atlases across tissues, organisms, and disease states provides an invaluable resource for cell type identification [1]. Hierarchical classification frameworks can leverage these atlas-scale references to automatically annotate new datasets while accounting for the hierarchical relationships between cell types. This approach will become increasingly powerful as atlases expand to include more conditions, developmental timepoints, and diverse patient populations.

In conclusion, accurate cell type identification remains a central challenge in scRNA-seq analysis, with implications for basic biological discovery and clinical translation. The hierarchical classification framework implemented in scClassify addresses key limitations of traditional approaches by incorporating ensemble learning, cell type hierarchies, sample size estimation, and support for multiple reference datasets [3]. As single-cell technologies continue to advance and generate increasingly complex datasets, such sophisticated computational frameworks will be essential for extracting meaningful biological and clinical insights from the breathtaking complexity of cellular systems.

scClassify is a multiscale classification framework designed for accurate cell type identification from single-cell RNA-sequencing (scRNA-seq) data. As supervised learning becomes increasingly important in scRNA-seq analysis, scClassify addresses key limitations of existing methods by incorporating ensemble learning and cell type hierarchies constructed from single or multiple annotated reference datasets [10] [3]. This approach enables researchers to automatically annotate cell types in new query datasets while accounting for hierarchical relationships between cell types and estimating required sample sizes for accurate classification [3].

The fundamental innovation of scClassify lies in its departure from traditional "one-step" classification approaches that ignore hierarchical relationships between cell types. By constructing a cell type tree from reference data where cell types are organized hierarchically with increasingly fine-tuned annotation, scClassify captures the natural biological progression from broad to specific cell types [3] [11]. This hierarchical organization, combined with ensemble learning, allows scClassify to consistently outperform other supervised cell type classification methods across diverse datasets representing different sizes, technologies, and complexity levels [10] [3].

Theoretical Framework and Algorithmic Foundations

Hierarchical Cell Type Classification

scClassify employs a sophisticated hierarchical framework that mirrors the biological relationships between cell types. The system utilizes the Hierarchical Ordered Partitioning and Collapsing Hybrid (HOPACH) algorithm to construct a cell type tree from reference datasets [11]. Unlike standard hierarchical clustering, HOPACH allows a parent node to be partitioned into multiple child nodes, better representing the natural progression from broad to specific cell types where a cell type can have two or more subtypes [11].

The classification process follows a top-down approach through this hierarchy. Starting from the root node containing all cell types, scClassify calculates distances between a query cell and reference cells at each branch node. The cell progresses down the hierarchy only when two criteria are met: (1) the nearest neighbor cells have correlations higher than a threshold determined by a mixture model, and (2) the weights of its assigned cell type exceed a default threshold of 0.7 [11]. Cells that cannot progress beyond the root are labeled "unassigned," while those classified at branch nodes but not at leaves are considered to have intermediate cell types [3] [11].

Ensemble Learning Architecture

scClassify incorporates ensemble learning through a weighted k-nearest neighbor (kNN) classifier that combines multiple similarity metrics and gene selection methods. The framework employs six similarity metrics (Pearson's correlation, Spearman's correlation, Kendall's rank correlation, cosine distance, Jaccard distance, and weighted rank correlation) and five gene selection methods (differentially expressed genes, differentially variable genes, differentially distributed genes, differentially proportioned genes, and bimodally distributed genes) [11].

This combination generates 30 base classifiers, each using a different pairing of similarity metrics and gene selection methods. The ensemble classifier weights individual classifiers based on their training error using an AdaBoost-like approach [11]. The weight for each classifier is calculated as ( wt = \ln((1 - \epsilont)/\epsilont) ), where ( \epsilont ) is the error rate of base classifier ( t ). Classifiers with accuracy below 50% receive negative weight, effectively excluding poor performers from the final prediction [11].

Table 1: scClassify Ensemble Classifier Components

Component Type	Specific Methods	Function
Similarity Metrics	Pearson, Spearman, Kendall correlation; Cosine, Jaccard distance; Weighted rank correlation	Measure cell-to-cell similarity from different statistical perspectives
Gene Selection Methods	Differential Expression (DE), Differential Variability (DV), Differential Distribution (DD), Differential Proportion (DP), Bimodal Distribution (BI)	Identify informative genes for cell type discrimination using different selection criteria
Algorithm	Weighted k-Nearest Neighbor (WKNN)	Classification algorithm that weights nearer neighbors more heavily

Sample Size Estimation

A unique feature of scClassify is its ability to estimate the number of cells required in a reference dataset to accurately discriminate between cell types at any level in the hierarchy. The method fits an inverse power law to estimate the relationship between sample size and classification accuracy, enabling researchers to design experiments with sufficient cells for nuanced cell type identification [3]. This sample size estimation is particularly valuable for experimental design, ensuring reference datasets contain adequate cells for reliable classification [3].

Implementation Protocols

Data Preparation and Installation

To implement scClassify, begin by installing the package from Bioconductor using R:

The package requires log-transformed, size-factor normalized expression matrices where rows represent genes and columns represent cells for both reference and query datasets [12] [13]. The example below demonstrates loading sample pancreas datasets from Wang et al. and Xin et al.:

After loading data, examine cell type compositions to understand reference and query datasets:

Basic Classification Workflow

For non-ensemble scClassify using a single combination of parameters:

The cell type tree generated from reference data can be visualized using:

Prediction results are accessed via:

Ensemble Classification Protocol

For improved accuracy, implement ensemble classification with multiple similarity metrics:

With weighted_ensemble = TRUE (default), base classifiers are weighted by their accuracy rates in the reference data. Setting this to FALSE assigns equal weight to all classifiers [12] [13].

Training Custom Models

Researchers can train custom scClassify models for repeated use:

The resulting trainClass object contains the trained model that can be applied to multiple query datasets without retraining [12] [13].

Workflow Visualization

scClassify Hierarchical Classification Workflow

Performance Benchmarks and Validation

Comparative Performance Analysis

scClassify has been rigorously validated against 14 other supervised cell type classification methods across 114 pairs of reference and testing data [3]. These benchmarks represent diverse combinations of dataset sizes, technologies, and complexity levels. In these evaluations, scClassify consistently achieved higher accuracy than alternative methods, with the performance advantage being more pronounced in challenging cases where test data contained cell types not present in the training data [3].

Table 2: scClassify Performance Benchmarks Across Dataset Types

Dataset Type	Comparison Methods	scClassify Performance Advantage	Key Findings
Human Pancreas (6 studies)	14 supervised methods	Higher accuracy in 16/16 "easy" cases and 14/14 "hard" cases	Average accuracy of 72-93% across parameter settings; ensemble outperformed best single classifier
PBMC Datasets	Multiple supervised methods	Superior performance at both coarse and fine classification levels	Improvement greater at fine-grained level (level 2) than coarse level (level 1)
Tabula Muris Atlas	Scalability assessment	Successfully identified previously unidentified subpopulations	Demonstrated applicability to large-scale single-cell atlases

In pancreas data benchmarks involving six studies, scClassify's ensemble approach achieved classification accuracy between 72-93% across different parameter settings [3]. Notably, the ensemble classifier typically outperformed even the best single model, demonstrating the value of combining multiple classification strategies [3].

Robustness and Stability Assessment

The robustness of scClassify has been evaluated through repeated resampling of training datasets, showing highly reproducible classification accuracy that remained highly concordant with results from full training datasets [3]. Hyperparameter analysis revealed that the choice of k in weighted kNN had minimal impact on performance, and dynamic threshold determination for correlation thresholds generally outperformed hard-coded thresholds, particularly in cases where test data contained cell types absent from training data [3].

Advanced Applications and Extensions

Multiple Reference Integration

scClassify supports joint classification using multiple reference datasets, which increases effective sample size for model training, improves classification accuracy, and reduces the number of unassigned cells [3]. This capability is particularly valuable when no single reference dataset contains all relevant cell types or when sample sizes in individual references are insufficient for reliable classification.

Novel Cell Type Discovery

For unassigned cells that cannot be classified to existing cell types in the reference, scClassify incorporates a post-hoc clustering procedure using a modified version of the SIMLR algorithm [11]. Following clustering, differential expression analysis identifies marker genes for each cluster, enabling annotation based on known markers and discovery of potentially novel cell types [11].

scClassify2: Advanced Cell State Identification

The scClassify framework has been extended to scClassify2, which specifically addresses the challenge of identifying adjacent cell states in continuous biological processes [14]. This advancement incorporates:

• Dual-layer architecture that integrates expression information and gene co-expression patterns derived from log-ratio of genes [14]
• Message Passing Neural Networks (MPNN) to capture both node and edge information in gene expression networks [14]
• Ordinal regression to effectively model sequential cell state transitions [14]

In benchmarking across eight diverse datasets, scClassify2 demonstrated significant improvement over the original scClassify, with accuracy increasing from 67.22% to 80.76% on dataset 8 [14]. The method also outperformed other state-of-the-art approaches including scGPT and scFoundation on most test datasets [14].

Research Reagent Solutions

Table 3: Essential Research Resources for scClassify Implementation

Resource Type	Specific Resource	Function in Experimental Workflow
Pre-trained Models	Mouse Primary Visual Cortex (Tasic 2018), Human Liver (MacParland), Human Pancreas (Multiple studies), Tabula Muris Atlas	Reference models for specific tissues and organisms; accelerate analysis without requiring training
Software Packages	scClassify R/Bioconductor package, Shiny web application (beta)	Core implementation; interactive interface for non-programmers
Reference Datasets	Gene Expression Omnibus (GEO) accessions: GSE115746, GSE84133, GSE109774, E-MTAB-5061	Standardized benchmarking; model training and validation
Gene Annotation Resources	Mm Gene Symbol, Hs Gene Symbol, ENSEMBL ID	Gene identifier conversion; cross-species comparison

The scClassify platform provides numerous pre-trained models for various tissues and organisms, readily available for download [15]. These include models for mouse primary visual cortex (Tasic 2018 and 2016), mouse visual cortex (Hrvatin), mouse lung (Cohen), mouse kidney (Park), human liver (MacParland and Aizarani), human pancreas (multiple studies), human melanoma (Li), PBMC (Ding), and the comprehensive Tabula Muris atlas [15].

Technical Specifications and Integration

Computational Requirements and Scalability

In terms of computational efficiency and memory requirements, scClassify performs comparably to other existing methods and can be applied to datasets with large numbers of cells [3]. Evaluation using the Tabula Muris dataset with varying numbers of cells or cell types demonstrated practical scalability for typical single-cell studies [3].

Integration with Single-Cell Analysis Pipelines

scClassify integrates seamlessly with standard single-cell analysis workflows, accepting log-transformed, size-factor normalized expression matrices compatible with outputs from preprocessing tools like Seurat, Scanpy, and scran [12] [13]. The package also provides functions to convert results to formats compatible with visualization tools, facilitating downstream biological interpretation.

scClassify represents a significant advancement in automated cell type identification from scRNA-seq data by addressing critical limitations in existing supervised methods. Through its multiscale classification framework based on ensemble learning and cell type hierarchies, scClassify enables more accurate, robust, and biologically informed cell type annotation. The implementation of sample size estimation further enhances its utility for experimental design. With the recent introduction of scClassify2 for identifying adjacent cell states, the framework continues to evolve to address emerging challenges in single-cell transcriptomics. As the collection of well-annotated scRNA-seq datasets continues to grow, scClassify provides an essential tool for leveraging these resources to automate and improve cell type identification in new studies.

Cell type annotation represents a fundamental prerequisite for downstream biological exploration in single-cell transcriptomics research. Within this domain, hierarchical classification has emerged as a powerful strategy for organizing cellular identities in a biologically meaningful structure. The scClassify framework implements this approach by constructing a cell type hierarchical tree through a recursive clustering algorithm, enabling systematic organization of cellular identities from broad categories to specific subtypes. This methodology is particularly valuable for capturing sequential cell states, which can be annotated under intermediate cell type categories, providing a nuanced understanding of cellular differentiation and transition states [14].

The hierarchical approach addresses critical challenges in single-cell analysis, where cell expression states form a continuous space rather than distinct clusters. Although differentiation follows continuous trajectories, cells within these trajectories can be effectively annotated as discrete but sequential states, a task for which hierarchical structures are uniquely suited. This innovation has proven particularly relevant for biological systems involving cell transitions, such as human preimplantation embryo development and T cell differentiation during infection, where adjacent cell states exhibit high similarity and often result in overlapping clustering [14].

Core Methodological Framework

Hierarchical Tree Construction

The scClassify framework employs a recursive clustering algorithm to construct cell type hierarchies that mirror biological relationships. This process begins with the identification of broad cell classes, which are subsequently divided into progressively finer subtypes based on transcriptional similarity. The hierarchical structure enables the model to capture relationships between cell types at multiple resolutions, from major lineages to finely resolved states [14].

The algorithm leverages gene expression patterns to establish phylogenetic relationships between cell populations, organizing them in a tree structure where branch lengths represent transcriptional distances. This organization allows for precise annotation of query cells by traversing the hierarchy from root to leaf nodes, comparing cellular profiles at each decision point to determine the most specific assignable identity [14].

Advanced Architectures for State Identification

Recent innovations have extended the hierarchical approach through scClassify2, which introduces a dual-layer architecture incorporating message passing neural networks (MPNN). This architecture integrates two levels of biological information: (1) log-ratio of pairwise gene expression counts modeled as edges, and (2) biological knowledge derived from gene co-expression patterns modeled as nodes. The MPNN framework allows information to propagate among genes across connecting edges, capturing subtle gene expression topology that characterizes different cell states [14].

A critical advancement in scClassify2 is the incorporation of ordinal regression for identifying adjacent cell states in sequential processes. Unlike conventional multi-class classification that treats all cell states as independent categories, ordinal regression explicitly models the sequential relationships between transitional states. This approach significantly improves identification of intermediate states, with experiments demonstrating accuracy improvements from 0.82 with conventional classification to 0.93 with ordinal regression for mouse gastrulation embryonic development cell states [14].

Table 1: Performance Comparison of Classification Approaches on Sequential Cell States

Classification Method	Dataset	Accuracy	Key Strength
scClassify2 with Ordinal Regression	Mouse Gastrulation	93%	Captures sequential relationships
Conventional Multi-classification	Mouse Gastrulation	82%	Standard approach
scClassify2	Dataset 3	87.93 ± 0.28%	Dual-layer architecture
sigGCN	Dataset 3	78.55 ± 0.34%	Graph neural network
scGCN	Dataset 3	79.31 ± 1.13%	Graph neural network
scGPT	Dataset 1	93.04 ± 0.18%	Large language model
scFoundation	Dataset 1	91.06 ± 0.10%	Foundation model

Experimental Protocols and Implementation

Protocol: Constructing Cell Type Hierarchies

Purpose: To establish a hierarchical classification system for cell identity annotation that captures biological relationships at multiple resolutions.

Materials and Reagents:

• Single-cell RNA sequencing data (count matrix)
• Reference dataset with annotated cell types
• Computational environment with R/Python and scClassify package

Procedure:

• Data Preprocessing: Normalize raw count data using standard scRNA-seq preprocessing pipelines. Select highly variable genes focusing on those with consistent expression patterns across datasets.
• Reference Tree Construction:
  • Apply recursive clustering to reference data using the buildTree function in scClassify
  • Determine optimal clustering resolution at each hierarchy level using internal validation metrics
  • Annotate each node in the hierarchy with cell type labels based on marker genes
• Feature Selection:
  • Implement scClassify's feature selection method that identifies genes with stable hierarchical discrimination power
  • Prioritize genes that maintain discriminative power across multiple hierarchy levels
• Model Training:
  • Train classifier ensembles for each node in the hierarchy
  • Optimize parameters using cross-validation on reference data
  • Validate hierarchy consistency ensuring child nodes represent legitimate subtypes of parent classifications
• Query Annotation:
  • Project query cells onto the reference hierarchy starting from the root node
  • At each node, apply the corresponding classifier to determine appropriate child branch
  • Continue until reaching leaf nodes or classification confidence falls below threshold

Validation: Assess hierarchical annotation accuracy using cross-validation on reference data. Evaluate biological consistency of the hierarchy through enrichment analysis of cell-type-specific marker genes at each node [14].

Protocol: Hierarchical Classification of Adjacent Cell States

Purpose: To precisely identify sequential cell states during differentiation processes using hierarchical classification with ordinal regression.

Materials and Reagents:

• scRNA-seq data from time-course or differentiation experiments
• Prior knowledge of developmental trajectory
• scClassify2 implementation with MPNN capability

Procedure:

• Experimental Design:
  • Collect cells spanning the entire differentiation continuum
  • Establish ground truth ordering of cell states based on experimental time points or pseudotime analysis
• Dual-Layer Graph Construction:
  • Represent each cell as a graph with genes as nodes
  • Calculate log-ratio of pairwise gene expressions as edge weights
  • Incorporate biological knowledge using Gene2Vec embeddings for node features
• Message Passing Neural Network Configuration:
  • Implement MPNN architecture with graph convolution layers
  • Configure message functions to combine node and edge features
  • Set up update functions to propagate information across the graph
• Ordinal Regression Layer:
  • Implement ordinal output layer with conditional probability distributions between adjacent states
  • Configure loss function to penalize misclassifications proportional to ordinal distance
• Model Training:
  • Train MPNN with ordinal regression using cells with known state assignments
  • Employ early stopping based on validation set performance
  • Assess model calibration ensuring confidence scores reflect true probabilities
• Hierarchical State Annotation:
  • Apply trained model to query cells
  • Generate probability distributions across ordered states
  • Assign final state based on maximum probability with confidence estimation

Validation: Quantify accuracy using cross-validation stratified across biological replicates. Compare with non-hierarchical approaches specifically examining performance on intermediate states [14].

Visualization of Hierarchical Classification Framework

Hierarchical Classification Framework with Dual-Layer Architecture

Performance Benchmarking and Quantitative Assessment

The hierarchical approach implemented in scClassify2 demonstrates competitive performance against state-of-the-art methods across diverse datasets. Comparative analyses across eight benchmark datasets reveal that scClassify2 consistently outperforms other graph-neural-network-based methods, including sigGCN and scGCN, while showing slight advantages over the latest generative AI approaches like scGPT and scFoundation in most test datasets [14].

Table 2: Comprehensive Performance Evaluation Across Multiple Datasets

Method	Dataset 1 Accuracy	Dataset 3 Accuracy	Dataset 8 Accuracy	Key Innovation
scClassify2	94.45 ± 0.17%	87.93 ± 0.28%	80.76 ± 0.43%	Hierarchical + Dual-layer MPNN
scClassify (previous)	N/A	N/A	67.22 ± 0.82%	Hierarchical tree only
sigGCN	N/A	78.55 ± 0.34%	N/A	Graph neural network
scGCN	N/A	79.31 ± 1.13%	N/A	Graph neural network
scGPT	93.04 ± 0.18%	N/A	N/A	Large language model
scFoundation	91.06 ± 0.10%	N/A	N/A	Foundation model

The integration of biological knowledge through the dual-layer architecture provides significant performance enhancements. As demonstrated in experimental evaluations, the use of distributed gene representations (e.g., Gene2Vec) as node embeddings improves cell state identification accuracy from 0.86 with one-hot vectors to 0.95 with learned representations. This highlights the value of incorporating prior biological knowledge into the hierarchical classification framework [14].

Research Reagent Solutions

Table 3: Essential Research Reagents and Computational Tools

Reagent/Tool	Function	Application Context
scClassify Software Package	Hierarchical cell type classification	Annotation of scRNA-seq data using reference hierarchies
Gene2Vec Embeddings	Distributed gene representations	Capturing gene co-expression patterns for node features
Message Passing Neural Network (MPNN)	Graph-based deep learning	Integrating node and edge features in cellular graphs
Ordinal Regression Layer	Sequential state classification	Identifying adjacent cell states in differentiation processes
Recursive Clustering Algorithm	Hierarchy construction	Building cell type trees from reference data
Log-Ratio Expression Metrics	Cross-platform stable features	Calculating edge weights in cellular graphs
Reference Cell Atlas	Training data	Well-annotated datasets for hierarchy construction
Single-cell RNA sequencing Data	Experimental input	Gene expression matrices from platforms like 10X Genomics

Integration with Emerging Technologies

The hierarchical classification framework demonstrates compatibility with cutting-edge computational approaches, including integration with large language models. While LLM-based tools like LICT (Large Language Model-based Identifier for Cell Types) offer alternative annotation strategies, hierarchical methods provide structured biological context that complements these approaches. The multi-model integration strategy employed by LICT, which combines multiple LLMs to reduce uncertainty and increase annotation reliability, shares philosophical alignment with hierarchical classification's goal of robust annotation [16].

Emerging feature selection methodologies like PHet (Preserving Heterogeneity) further enhance hierarchical classification by identifying heterogeneity-preserving discriminative features that maintain sample heterogeneity while distinguishing known disease or cell states. These features enable more refined clustering of cells, facilitating deeper comprehension of heterogeneity factors that can be incorporated into hierarchical frameworks [17].

The hierarchical approach also shows exceptional generalizability across experimental platforms. scClassify2 has demonstrated effectiveness not only with single-cell RNA-sequencing data but also with subcellular spatial transcriptomics data, highlighting the transferability of the hierarchical classification principle across technological domains [14].

Concluding Remarks

The construction and leveraging of cell type hierarchies represents a fundamental innovation in single-cell bioinformatics, providing a biologically intuitive framework for organizing cellular diversity. The evolution from simple hierarchical trees to sophisticated dual-layer architectures with message passing neural networks demonstrates how biological knowledge can be systematically integrated into computational frameworks to enhance annotation accuracy, particularly for challenging scenarios involving sequential cell states.

As single-cell technologies continue to evolve, producing increasingly complex and multimodal datasets, hierarchical classification approaches will remain essential for extracting biologically meaningful insights from high-dimensional data. The integration of these approaches with emerging artificial intelligence methodologies promises to further advance our understanding of cellular identity and function in health and disease.

scClassify is a multiscale classification framework based on ensemble learning and cell type hierarchies constructed from single or multiple annotated single-cell RNA-sequencing (scRNA-seq) datasets as references [10]. This tool addresses key computational challenges in automated cell type identification by enabling sample size estimation for accurate classification and allowing joint classification when multiple references are available [10] [18]. The methodology represents state-of-the-art capability in automated cell type identification, consistently outperforming other supervised classification methods across diverse datasets varying in size, technology, and complexity [10].

The development of scClassify capitalizes on the growing collection of well-annotated scRNA-seq datasets, providing researchers with a robust framework for cell type identification that accommodates the inherent complexities of single-cell data. By implementing a hierarchical approach, scClassify mirrors biological relationships between cell types, creating a structured classification system that improves accuracy and interpretability compared to flat classification methods.

Theoretical Framework and Key Components

Hierarchical Classification Structure

The foundational innovation of scClassify lies in its multiscale classification framework that utilizes cell type hierarchies constructed from reference datasets. This hierarchical approach reflects the natural biological relationships between cell types, where broader categories branch into increasingly specific subtypes. The system employs ensemble learning methods to strengthen classification accuracy and robustness across different levels of the hierarchy [10]. This structure allows the algorithm to make classification decisions at multiple resolutions, from major cell lineages to finely resolved subtypes, providing flexibility depending on the biological question and data quality.

The hierarchical organization enables more biologically plausible classification, as cells are first assigned to broad categories before being refined into more specific types. This approach mimics the developmental relationships between cell types and can improve accuracy by leveraging shared characteristics within lineages. The ensemble learning component further enhances performance by combining multiple classification approaches or models, reducing the likelihood of errors from any single method and increasing overall reliability.

Sample Size Estimation Methodology

A critical innovation of scClassify is its integrated sample size estimation for accurate cell type classification within a cell type hierarchy [10]. This functionality addresses a fundamental challenge in experimental design - determining the number of cells needed to reliably identify cell types present in a sample. The sample size estimation feature provides researchers with guidance on the cellular input requirements for achieving robust classification results, supporting more rigorous experimental planning and resource allocation.

The sample size estimation accounts for factors such as the complexity of the cell type hierarchy, the distinguishability of different cell types based on their gene expression profiles, and the expected variability within cell populations. By providing these estimates, scClassify helps prevent underpowered studies that might miss rare cell types or fail to adequately resolve closely related subtypes, while also avoiding unnecessary oversampling that would increase sequencing costs without substantial informational benefit.

scClassify provides unique capability for joint classification of cells when multiple reference datasets are available [10]. This functionality addresses the challenge of leveraging multiple existing annotated datasets to classify a new query dataset, potentially incorporating complementary information from different sources. The multiple reference approach can enhance classification accuracy and coverage, particularly when individual references might be incomplete or biased in their cell type representation.

The joint classification with multiple references allows integration of knowledge from different experimental conditions, technologies, or biological contexts, creating a more comprehensive classification system than would be possible with any single reference. This capability is particularly valuable as the number of publicly available scRNA-seq datasets continues to grow, enabling researchers to build upon previous work rather than creating new references from scratch for each new study.

Quantitative Performance Metrics

Table 1: Performance evaluation of scClassify across diverse testing scenarios

Evaluation Metric	Performance Range	Testing Conditions	Comparative Advantage
Overall Accuracy	Consistently superior to other methods	114 reference-testing pairs [10]	Outperforms across diverse technologies and complexities
Scalability	Demonstrated on large single-cell atlases	Tabula Muris data [10]	Identified previously unrecognized subpopulations
Reference Flexibility	Single and multiple reference integration	Various ensemble configurations [10]	Enables knowledge integration from multiple sources
Sample Size Estimation	Integrated estimation capability	Various cell type hierarchies [10]	Guides experimental design for classification tasks

Implementation Protocols

Experimental Workflow for Hierarchical Classification

The following workflow diagram illustrates the complete experimental procedure for hierarchical classification with scClassify, encompassing both sample size estimation and joint classification with multiple references:

Sample Size Estimation Protocol

• Input Preparation: Prepare reference data with validated cell type annotations and query dataset for analysis.
• Hierarchy Construction: Generate cell type hierarchy based on biological relationships and transcriptional similarities.
• Parameter Configuration:
  • Set statistical power threshold (default: 80%)
  • Define significance level (default: α = 0.05)
  • Specify expected effect size based on pilot data or prior knowledge
• Estimation Execution: Run sample size estimation algorithm across hierarchy levels.
• Output Interpretation: Review recommended cell numbers per cell type for adequate classification power.

The sample size estimation employs statistical methods that account for the hierarchical structure of cell types, with different requirements at various levels of resolution. Broader categories (e.g., immune cells vs. epithelial cells) typically require fewer cells for reliable identification, while distinguishing closely related subtypes (e.g., T cell subsets) demands larger sample sizes to achieve sufficient statistical power.

• Reference Selection: Curate multiple annotated reference datasets with relevant cell types.
• Data Harmonization: Apply batch correction methods to address technical variations between references.
• Classifier Training: Train ensemble classifiers on each reference dataset independently.
• Classification Integration:
  • Apply all trained classifiers to query data
  • Implement consensus approach to reconcile classifications
  • Resolve conflicts using hierarchical relationships
• Result Synthesis: Generate unified cell type assignments leveraging complementary information from all references.

The joint classification protocol specifically addresses challenges such as conflicting classifications between references, incomplete cell type representation across references, and batch effects. The ensemble approach weights classifications based on reference quality and specificity, with conflict resolution mechanisms that prioritize higher-resolution assignments when supported by sufficient evidence.

Research Reagent Solutions

Table 2: Essential research reagents and computational tools for scClassify implementation

Reagent/Tool	Function	Implementation Notes
scClassify R Package	Core classification engine	Available through Bioconductor (version 1.23.0) [18]
Annotated Reference Datasets	Training data for classifiers	Curated from public repositories (e.g., Tabula Muris)
Single-Cell Analysis Tools	Data preprocessing and normalization	Compatible with Seurat, SingleCellExperiment objects
Batch Correction Algorithms	Harmonizing multiple references	Essential for joint classification protocols
Visualization Packages	Result interpretation and validation	UMAP, t-SNE, hierarchical dendrograms

Application in Drug Development Context

The hierarchical classification approach implemented in scClassify aligns with Model-Informed Drug Development (MIDD) frameworks, particularly in target identification and lead compound optimization stages [19]. In pharmaceutical development, precise cell type identification enables:

• Target Identification: Comprehensive characterization of cell types expressing therapeutic targets across tissues
• Toxicity Assessment: Identification of cell type-specific toxicities through precise classification of affected populations
• Biomarker Discovery: Detection of cell subpopulations associated with treatment response
• Mechanistic Studies: Resolution of drug effects on specific cell types in complex tissues

The sample size estimation component of scClassify supports rigorous experimental design in preclinical studies, ensuring adequate power for detecting biologically relevant cell populations affected by therapeutic interventions. This statistical rigor enhances the reliability of conclusions drawn from single-cell studies in drug development pipelines.

Advanced Implementation Considerations

Technical Validation and Quality Control

• Classification Confidence: Assess assignment confidence scores at each hierarchy level
• Cross-Validation: Implement stratified cross-validation respecting hierarchical structure
• Stability Testing: Evaluate classification consistency across subsampled data
• Reference Quality Metrics: Quantify reference dataset suitability for classification tasks

Optimization Strategies

• Feature Selection: Identify optimal gene sets for different classification levels
• Parameter Tuning: Optimize hierarchy-specific parameters for sample size estimation
• Computational Efficiency: Implement parallel processing for large-scale applications
• Memory Management: Employ efficient data structures for large reference collections

scClassify provides a comprehensive framework for hierarchical cell type classification with integrated sample size estimation and support for multiple references. The methodology offers robust performance across diverse datasets and experimental conditions, addressing critical challenges in single-cell genomics analysis. The protocols outlined enable researchers to implement these approaches effectively, supporting rigorous experimental design and comprehensive cell type identification. As single-cell technologies continue to advance and reference datasets expand, hierarchical classification approaches will play an increasingly important role in extracting biological insights from complex cellular systems.

scClassify is a single-cell RNA sequencing (scRNA-seq) classification package that implements a set of methods to perform accurate cell type classification based on ensemble learning and sample size calculation [12]. A fundamental task in single-cell research is cell annotation, as understanding the identity of cells is key to further downstream analysis [20]. While many approaches for cell type annotation exist, a significant portion focuses on discrete and non-sequential cell subpopulations, overlooking the challenge of identifying adjacent cell states that are typically more similar as they represent transitions from one to the other [20]. The scClassify framework addresses this gap through its hierarchical approach, positioning itself as a crucial tool for researchers and drug development professionals working with cellular heterogeneity.

Core Methodologies and Experimental Protocols

Hierarchical Tree Construction with HOPACH

The foundation of scClassify's hierarchical classification approach is the construction of a cell type tree through HOPACH (Hierarchical Ordered Partitioning and Collapsing Hybrid), a recursive clustering algorithm that captures sequential cell states by annotating them under intermediate cell type categories [20] [12].

Protocol: Building Cell Type Hierarchies

• Input Preparation: Provide log-transformed, size-factor normalized expression matrices where each row represents a gene and each column represents a cell
• Feature Selection: Utilize differential expression analysis ("limma") to identify informative genes for tree construction
• Tree Construction: Execute HOPACH algorithm to recursively cluster cell types based on molecular characteristics
• Tree Visualization: Generate hierarchical representations of cell type relationships using plotCellTypeTree(cellTypeTree(scClassify_res$trainRes)) [12]

Non-Ensemble Classification Protocol

The basic scClassify implementation involves using a reference dataset to classify cells in a query dataset through similarity measurement.

Protocol: Basic Cell Type Classification

• Reference Data Preparation: Log-transform and normalize reference scRNA-seq data with known cell type annotations
• Query Data Processing: Apply identical normalization procedures to query dataset
• Feature Selection: Identify differentially expressed genes using limma-based methods
• Similarity Calculation: Compute Pearson correlations between query cells and reference cell types
• Weighted K-Nearest Neighbors (WKNN): Classify cells based on highest similarity scores [12]

Code Implementation:

Ensemble Classification for Improved Accuracy

The ensemble approach combines multiple classifiers to enhance prediction robustness and accuracy.

Protocol: Ensemble scClassify Implementation

• Multiple Classifier Generation: Create base classifiers with different similarity metrics (Pearson, cosine)
• Weighting Strategy: Assign weights to classifiers based on reference data accuracy (weighted_ensemble = TRUE) or equal weighting (weighted_ensemble = FALSE)
• Consensus Prediction: Aggregate predictions from all base classifiers
• Unassigned Cell Handling: Identify cells that cannot be confidently classified [12]

Code Implementation:

Advanced Protocol: scClassify2 for Sequential Cell State Identification

The recently developed scClassify2 extension specifically focuses on adjacent cell state identification through three key innovations [20].

Protocol: Dual-Layer Architecture with Message Passing Neural Networks

• Transferable Component: Implement reference-free markers by examining log-ratio of expression values to capture consistent relationships between genes
• Dual-Layer Design:
  • Layer 1: Process log-ratio of pairwise gene expression counts modeled as edges
  • Layer 2: Incorporate biological knowledge from gene co-expression patterns modeled as nodes
• Message Passing Neural Network (MPNN): Capture gene co-expression patterns and relationships between states using graph neural networks
• Ordinal Regression: Implement conditional training procedure to identify adjacent cell state transitions [20]

Performance Evaluation and Quantitative Assessment

Comparative Performance Across Datasets

Table 1: scClassify2 Performance Comparison with State-of-the-Art Methods [20]

Dataset	scClassify2	scClassify	sigGCN	scGCN	scGPT	scFoundation
Dataset 1	87.93 ± 0.28%	82.15%	78.55 ± 0.34%	79.31 ± 1.13%	86.20%	85.95%
Dataset 3	89.45%	81.33%	82.10%	80.75%	88.90%	88.12%
Dataset 8	80.76 ± 0.43%	67.22 ± 0.82%	72.18%	71.45%	79.80%	79.25%

Impact of Architectural Components

Table 2: Component-Wise Performance Contribution in scClassify2 [20]

Component	Configuration	Accuracy	Improvement Basis
Biological Information	No information (zero vectors)	0.63	Baseline
	With biological information (one-hot vectors)	0.86	+36.5%
Gene Representation	One-hot vectors	0.86	Baseline
	Gene2vec embeddings	0.95	+10.5%
Classification Method	Conventional multi-classification	0.82	Baseline
	Ordinal regression	0.93	+13.4%

Table 3: Key Research Reagent Solutions for scClassify Implementation

Resource	Type	Function	Application Context
scClassify R Package	Software Library	Implements core classification algorithms with ensemble learning	Cell type annotation from scRNA-seq data [12]
HOPACH Algorithm	Computational Method	Constructs cell type hierarchical trees through recursive clustering	Capturing sequential cell states and relationships [20] [12]
Gene2Vec Embeddings	Pre-trained Model	Provides distributed gene representations capturing co-expression patterns	Enhancing cell state identification accuracy from 0.86 to 0.95 [20]
Message Passing Neural Network (MPNN)	Graph Neural Network	Incorporates both node and edge information for subtle pattern recognition	Dual-layer architecture for adjacent cell state identification [20]
Ordinal Regression Layer	Machine Learning Component	Captures sequential nature between transitional cell states	Identifying adjacent cell states in developmental processes [20]
Web Server Catalogue	Online Resource	Provides pre-trained models for various human tissues	Community resource for standardized cell state annotations [20]

Integrated Workflow for Hierarchical Cell Classification

Advanced Applications in Pharmacological Profiling

The principles underlying scClassify have been extended to pharmacological applications through models like scGSDR (Single-cell Gene Semantics for Drug Response prediction), which employs a dual computational pipeline to integrate prior knowledge of cellular states and gene signaling pathways [21]. This approach enhances predictive modeling of cellular responses to diverse drugs by incorporating gene semantics, proving invaluable for scenarios involving both single drug and combination therapies [21]. The methodology shares scClassify's emphasis on biological interpretability, using attention mechanisms to identify pathways contributing to drug-resistant and drug-sensitive phenotypes.

From Theory to Practice: A Step-by-Step Guide to Implementing scClassify

scClassify is a multiscale classification framework for single-cell RNA-seq data based on ensemble learning and cell type hierarchies [18]. It enables sample size estimation for accurate cell type classification and joint classification of cells using multiple references [3]. To install scClassify via Bioconductor, specific R version compatibility must be considered, as different Bioconductor versions require different R versions.

Table: Bioconductor Version Compatibility

Bioconductor Version	Required R Version	Installation Command
Development (3.23)	R (≥ 4.6)	BiocManager::install(version='devel') followed by BiocManager::install("scClassify")
Release (3.20)	R (≥ 4.4)	BiocManager::install("scClassify")
Historical (3.11)	R (≥ 4.0.0)	BiocManager::install("scClassify")

The installation process begins by ensuring BiocManager is available, which facilitates the installation of Bioconductor packages [18] [22]:

For those needing the development version of Bioconductor, additional steps are required to initialize the usage of Bioconductor devel before package installation [18].

Initial Setup and Data Preparation

After successful installation, load the package and prepare your single-cell RNA-seq data. scClassify requires log-transformed, size-factor normalized expression matrices where rows represent genes and columns represent cells [12]. The example dataset below demonstrates the typical data structure required:

Table: Example Dataset Composition

Dataset	Cell Types	Number of Cells	Cell Type Distribution
Xin et al.	4 types	674	alpha (285), beta (261), delta (49), gamma (79)
Wang et al.	7 types	501	acinar (5), alpha (206), beta (118), delta (10), ductal (96), gamma (21), stellate (45)

Basic Classification Workflow

The core function scClassify() performs hierarchical classification using a reference dataset to predict cell types in a query dataset [12]. The basic implementation requires specifying the training and testing data, algorithm, feature selection method, and similarity metric:

The cell type hierarchy constructed from the reference data can be visualized to understand the classification structure [12]:

The prediction results show how query cells are classified, including any unassigned cells:

Ensemble Classification for Improved Accuracy

scClassify implements ensemble learning to combine multiple classifiers, improving accuracy over individual methods [3] [12]. Research has demonstrated that while individual classifiers show performance variation (accuracy range: 72-93%), ensemble classifiers typically achieve higher accuracy than single best models [3]. The ensemble approach can incorporate multiple similarity metrics:

Table: Ensemble Method Comparison

Parameter	Base Classifier 1	Base Classifier 2	Ensemble Result
Similarity metric	Pearson	Cosine	Combined prediction
Feature selection	limma	limma	limma
Algorithm	WKNN	WKNN	WKNN
Weighting	Equal weight	Equal weight	Performance-based or equal

Training Custom Models and Using Pretrained Models

Training Custom Models

Users can train custom scClassifyTrainModel objects using train_scClassify(), which stores the reference data, feature selection results, and cell type hierarchy [12]:

Using Pretrained Models

scClassify supports using pretrained models for cell type prediction, available through the package's resource page [23]. This approach saves computational time by leveraging existing trained models:

Experimental Design and Workflow

The following diagram illustrates the complete scClassify workflow, from data preparation through to hierarchical classification and result interpretation:

Key Research Reagents and Computational Solutions

Table: Essential Components for scClassify Implementation

Component	Type	Function	Example/Value
Reference Data	Biological Data	Training set with validated cell types	Xin et al. pancreas dataset (674 cells)
Query Data	Biological Data	Unknown cells for classification	Wang et al. dataset (501 cells)
Normalization Method	Computational Step	Data preprocessing	Log-transformed size-factor normalized counts
HOPACH Tree	Algorithm	Cell type hierarchy construction	Hierarchical clustering of cell types
limma	Feature Selection	Differential expression analysis	Gene selection for classification
WKNN	Algorithm	Weighted k-nearest neighbors	Cell type prediction
Pearson/Spearman	Similarity Metric	Distance measurement between cells	Correlation-based classification
Ensemble Framework	Method	Combine multiple classifiers	Improved accuracy over single methods

Performance and Validation

Benchmarking studies have demonstrated that scClassify consistently outperforms other supervised cell type classification methods across 114 pairs of reference and testing data, representing diverse combinations of sizes, technologies, and complexity levels [3]. The method shows particular advantage in challenging cases where test data contains cell types not present in the training data [3].

The package also includes functionality for sample size estimation, which helps researchers determine the number of cells required in reference datasets to accurately discriminate between cell types at any level in the hierarchy [3]. This feature uses an inverse power law to model the relationship between sample size and classification accuracy, supporting robust experimental design.

Proper data preparation is a critical first step for successful cell type identification using scClassify, a multiscale hierarchical classification framework for single-cell RNA-seq data. This protocol details the specific procedures for formatting your single-cell RNA sequencing (scRNA-seq) reference and query datasets to ensure accurate cell type classification. The quality of input data directly influences scClassify's ensemble learning performance and the biological validity of the resulting cell type hierarchies [3]. Within the broader context of hierarchical classification research, appropriate matrix formatting ensures that the algorithm can effectively construct meaningful cell type trees and leverage multiple similarity metrics for robust prediction [12] [3].

Data Formatting Specifications

Input Matrix Requirements

scClassify requires specific matrix formats to function correctly. The input data must adhere to the following specifications:

• Data Type: Log-transformed, size-factor normalized expression matrices [12]
• Matrix Orientation: Rows representing genes and columns representing cells [12]
• Format Compatibility: Both standard matrices and sparse matrix formats (dgCMatrix) are supported [12]

The example below demonstrates the proper matrix setup using pancreas datasets from independent studies:

Table 1: Key Specifications for Input Matrices

Parameter	Requirement	Example	Purpose
Normalization	Size-factor normalized & log-transformed	log2(counts/sf + 1)	Stabilizes variance & enables distance calculations
Matrix Format	dgCMatrix (preferred) or standard matrix	as(exprsMat, "dgCMatrix")	Efficient memory usage for sparse scRNA-seq data
Dimension Meaning	Rows = Genes, Columns = Cells	1000 genes × 500 cells	Standardized orientation for algorithm processing
Data Structure	Numeric expression values	Continuous, non-negative	Compatibility with correlation-based similarity metrics

Cell Type Annotation Formatting

Cell type annotations must be formatted as character vectors corresponding to the columns of the expression matrix:

Table 2: Cell Type Annotation Requirements

Component	Format	Description	Importance for Hierarchy
Labels Vector	Character vector	Named vector matching matrix columns	Provides ground truth for tree construction
Cell Type Names	Descriptive labels	e.g., "TcellCD4_naive"	Enables meaningful hierarchical relationships
Consistency	Uniform nomenclature	Consistent across reference & query	Facilitates accurate cross-dataset classification

Experimental Protocol for Data Preparation

Reference Dataset Formatting

The reference dataset serves as the training basis for scClassify's hierarchical model. Follow this standardized protocol:

Step 1: Normalization and Transformation

• Perform size-factor normalization using standard scRNA-seq processing tools
• Apply log-transformation (typically log2(CPM/TPM + 1))
• Remove genes with zero expression across all cells

Step 2: Matrix Formatting

Step 3: Quality Control

• Remove poor-quality cells (high mitochondrial percentage, low gene counts)
• Ensure cell type annotations match the matrix columns
• Check for batch effects that might confound classification

Query Dataset Preparation

Query datasets must align with the reference structure:

Step 1: Gene Space Matching

• Subset query genes to match reference gene space
• Maintain the same gene identifiers and order where possible
• Handle missing genes through appropriate imputation or exclusion

Step 2: Normalization Consistency

• Apply identical normalization procedures to both reference and query
• Ensure comparable expression value distributions
• Check for systematic technical differences between datasets

Step 3: Format Verification

Hierarchical Classification Data Flow

The data formatting directly enables scClassify's hierarchical classification approach. The following diagram illustrates how properly formatted matrices flow through the classification system:

Diagram 1: Data Flow in Hierarchical Classification - This workflow illustrates how properly formatted matrices enable scClassify's hierarchical classification, from data preparation through cell type prediction.

The Scientist's Toolkit

Table 3: Essential Research Reagent Solutions for scClassify Implementation

Tool/Resource	Function	Application in Protocol
Bioconductor scClassify Package	Core classification algorithms	Installation via BiocManager::install("scClassify") [12]
dgCMatrix Format	Sparse matrix representation	Efficient storage of single-cell expression data [12]
HOPACH Algorithm	Hierarchical tree construction	Organizes cell types into biologically meaningful hierarchies [12]
Pre-trained Models	Reference classification models	Accelerate analysis using curated datasets [15] [24]
limma Feature Selection	Differential expression analysis	Identifies informative genes for classification [12]
Multiple Similarity Metrics	Distance calculations	Ensemble learning using Pearson, cosine, Spearman correlations [12]

Advanced Data Considerations

Handling Multiple Reference Datasets

scClassify supports joint classification using multiple references, which requires additional formatting considerations:

Cell Type Tree Construction

The hierarchical structure is automatically built from reference data, but understanding this process informs data preparation:

Diagram 2: Cell Type Hierarchy Example - Proper data formatting enables scClassify to construct biologically meaningful cell type hierarchies that capture relationships between major lineages and subtypes.

Proper data preparation ensures that scClassify can leverage its full multiscale classification framework, including ensemble learning with multiple gene selection methods and similarity metrics, accurate cell type tree construction, and optimal handling of both easy cases (where all query cell types exist in the reference) and challenging cases (with novel cell types in query data) [3]. The hierarchical approach allows for sample size-appropriate classification, where cells may be assigned to intermediate non-terminal nodes when reference sample size is insufficient for subtype-level classification [3].

scClassify is a multiscale classification framework for single-cell RNA-seq (scRNA-seq) data based on ensemble learning and cell type hierarchies [18]. It enables accurate cell type classification by leveraging cell type hierarchies and ensemble learning strategies. This protocol details the use of the train_scClassify function to build a classification model, a critical step for annotating cell types in query datasets. This process is foundational to research in cellular composition and function, with direct applications in drug development and disease mechanism exploration.

Experimental Protocol & Workflow

The following diagram illustrates the end-to-end process of training and applying an scClassify model.

Detailed Methodology

Data Preprocessing and Feature Selection

Prior to model training, raw scRNA-seq data must undergo rigorous preprocessing.

• Quality Control (QC): Filter cells based on metrics like mitochondrial gene percentage and unique gene counts. Filter genes detected in very few cells.
• Normalization: Normalize gene expression counts to account for library size differences, using methods like log-normalization.
• Feature Selection: Identify Highly Variable Genes (HVGs) that contribute most to cell-to-cell variation. These genes form the feature set for classifier training. The scClassify package provides functions to integrate these preprocessing steps.

Thetrain_scClassifyFunction

The core training function builds a hierarchical model using ensemble learning.

• Input: The primary input is the preprocessed reference expression matrix with known cell type labels.
• Hierarchy Construction: The function automatically constructs a cell-type hierarchy by assessing the transcriptional similarities between different cell types. This tree-like structure groups closely related cell types.
• Ensemble Training: At each node of the hierarchy, an ensemble of classifiers is trained to distinguish between the cell types present. This multi-scale approach improves accuracy, especially for closely related or rare cell types.
• Output: The function returns a trained model object containing the hierarchy and all ensemble classifiers, ready for predicting cell types in new query data.

Model Validation and Interpretation

After training, the model's performance must be rigorously evaluated.

• Cross-Validation: Use internal cross-validation on the training data to estimate accuracy and prevent overfitting.
• Independent Validation: Apply the model to a held-out test dataset with known labels to compute performance metrics like accuracy, precision, and recall for each cell type.
• Visual Inspection: Use dimensionality reduction plots (e.g., UMAP, t-SNE) to visually confirm that predicted labels form coherent clusters.

Key Research Reagent Solutions

Table 1: Essential computational tools and resources for training and applying scClassify models.

Category	Item / Software	Function / Application
Software & Packages	R (v4.0+)	The programming language and environment for statistical computing in which scClassify operates [18].
	Bioconductor	The project repository from which the scClassify package is installed and managed [18].
	SingleCellExperiment	A standard S4 class for storing single-cell genomics data, often used as input for scClassify.
Data Resources	Reference scRNA-seq Datasets	Pre-annotated datasets (e.g., from human cell atlas projects) used to train accurate classification models.
	Single-cell RNA sequencing Data	The primary input data (both reference and query), containing gene expression counts for individual cells [20].
Supporting Tools	scGPT / Geneformer	Large language models for single-cell biology that can provide alternative or complementary cell representations [20].
	scClassify2	An advanced version of the framework specifically designed for identifying sequential cell states using message-passing neural networks (MPNN) and ordinal regression [20].

Performance Metrics and Benchmarking

To ensure the trained model is robust, its performance should be benchmarked against established metrics and other methods.

Quantitative Performance Evaluation

Table 2: Example performance of scClassify2 on sequential cell state identification tasks across diverse datasets.

Dataset	scClassify2 Accuracy	scGPT Accuracy	scGCN Accuracy	Key Challenge Addressed
Mouse Gastrulation Embryo	93%	Information Missing	82% (Multi-class)	Ordinal regression captures developmental sequence [20].
Dataset 1	~87% (Inferred)	~86% (Inferred)	Information Missing	Generalization across platforms [20].
Dataset 3	87.93 ± 0.28%	Information Missing	79.31 ± 1.13%	Distinguishing subtly different cell states [20].
Dataset 8	80.76 ± 0.43%	Information Missing	Information Missing	Handling complex cell state transitions [20].

Advanced Framework: scClassify2 for Sequential States

For complex tasks involving developmental trajectories or continuous processes, the scClassify2 framework is recommended. Its architecture is specifically designed to identify adjacent cell states, a known challenge in single-cell analysis [20]. The following diagram details its innovative dual-layer design.

Key Innovations of scClassify2:

• Dual-Layer Architecture: Integrates gene node embeddings (e.g., from Gene2Vec) with log-ratio of pairwise gene expressions. This captures stable gene-gene relationships across datasets, improving generalization [20].
• Message Passing Neural Network (MPNN): Allows information to propagate between genes, capturing complex gene co-expression topologies that define subtle cell states [20].
• Ordinal Regression: Replaces standard multi-class classification. It explicitly models the inherent order of transitional cell states (e.g., E6.75 -> E7.0 -> E7.25), drastically improving accuracy on sequential state identification tasks [20].

Troubleshooting and Technical Notes

• Data Quality is Paramount: The accuracy of the final model is heavily dependent on the quality of the reference data. Ensure careful curation, normalization, and annotation of the training set.
• Feature Selection Impact: The set of HVGs used for training is critical. Consider using gene sets derived from large-scale integration of multiple datasets (as in scClassify2) for more stable and generalizable features.
• Handling Ambiguous Cells: For cells in transitional states, the standard scClassify model may assign low-confidence probabilities. Using the scClassify2 framework with ordinal regression is specifically designed for this scenario and will yield more reliable annotations.
• Computational Resources: Training on large reference datasets with many cell types and genes can be computationally intensive. Utilize the built-in support for parallelization in scClassify via the BiocParallel package to reduce runtime [18].

scClassify is a multiscale classification framework based on ensemble learning and cell type hierarchies for automated cell type identification from single-cell RNA-sequencing (scRNA-seq) data. Unlike traditional "one-step" classification approaches that directly assign cells to terminal cell types, scClassify constructs a hierarchical cell type tree from reference datasets using the HOPACH algorithm, allowing for more nuanced classification that accounts for hierarchical relationships between cell types. This framework enables sample size estimation for accurate classification and permits joint classification when multiple reference datasets are available, significantly improving annotation accuracy compared to conventional supervised methods [3] [11].

The power of scClassify lies in its ensemble approach, which combines multiple weighted k-nearest neighbor classifiers using different similarity metrics and gene selection methods. This strategy captures diverse cell type characteristics and demonstrates consistently better performance across diverse datasets compared to other supervised cell type classification methods, as validated through extensive benchmarking across 114 pairs of reference and testing data representing various sizes, technologies, and complexity levels [3]. The hierarchical nature of the classification process also allows cells to be assigned to intermediate cell types when the reference dataset lacks sufficient sample size for definitive terminal classification, reducing misclassification errors.

Installation and Setup

Installation from Bioconductor

scClassify is available through Bioconductor and requires R version 4.0 or higher. Installation can be performed using the BiocManager package [18] [12]:

For those wishing to use the development version of the package, additional configuration is required:

Package Loading and Dependencies

Once installed, load the scClassify package into your R session:

scClassify imports several dependent packages including S4Vectors, limma, ggraph, igraph, cluster, minpack.lm, mixtools, BiocParallel, proxy, proxyC, Matrix, ggplot2, hopach, diptest, mgcv, and Cepo. These dependencies are automatically installed during the scClassify installation process and provide essential functionality for gene selection, tree construction, similarity calculations, and parallel processing [18].

Data Preparation Requirements

Input Data Formatting

scClassify requires log-transformed, size-factor normalized expression matrices where each row represents a gene and each column represents a cell. The package expects both reference (training) and query (test) datasets in this format, with consistent gene identifiers between datasets [12].

The input data should be provided as matrix objects, preferably in sparse matrix format (dgCMatrix) for computational efficiency with large datasets:

Example Dataset Structure

The scClassify package includes example datasets from pancreatic islet studies (Wang et al. and Xin et al.) that demonstrate the proper data structure [12] [25]:

These datasets illustrate the typical composition of scRNA-seq data for classification, with Xin et al. data containing 4 cell types (alpha, beta, delta, gamma) across 674 cells, and Wang et al. data containing 7 cell types (acinar, alpha, beta, delta, ductal, gamma, stellate) across 501 cells [12].

Core Classification Workflow

Basic Non-Ensemble Classification

The fundamental scClassify function requires reference expression data with cell type labels, and query data for prediction. A basic implementation uses WKNN (weighted k-nearest neighbors) as the algorithm, limma for differential expression gene selection, and Pearson correlation as the similarity metric [12]:

Table 1: Key Parameters for Basic scClassify Implementation

Parameter	Description	Options	Default
tree	Method for hierarchical tree construction	"HOPACH"	"HOPACH"
algorithm	Classification algorithm	"WKNN"	"WKNN"
selectFeatures	Gene selection method	"limma", "BI", "DV", "DD", "DP"	"limma"
similarity	Similarity metric	"pearson", "spearman", "cosine", etc.	"pearson"
returnList	Whether to return list format results	TRUE/FALSE	FALSE
verbose	Display progress messages	TRUE/FALSE	FALSE

Ensemble Classification Implementation

Ensemble classification integrates multiple classifiers with different gene selection methods and similarity metrics, typically yielding higher accuracy than individual classifiers [3] [12]:

The ensemble approach combines multiple base classifiers (up to 30 combinations of 6 similarity metrics and 5 gene selection methods) and weights them according to their training accuracy. Classifiers with less than 50% accuracy receive negative weights, ensuring only reliable predictors contribute positively to the final ensemble decision [11].

Custom Model Training

For advanced applications, users can train custom scClassify models for repeated use on multiple query datasets:

The resulting scClassifyTrainModel object contains the trained model hierarchy, selected features, and training data representation, which can be reused for classifying multiple query datasets without retraining [12].

Hierarchical Classification Process

Cell Type Tree Construction

The foundation of scClassify's hierarchical approach is the construction of a cell type tree using the HOPACH algorithm, which organizes cell types in a hierarchy with increasingly fine-grained annotation. The tree construction process begins with the union of differentially expressed genes identified using limma in one-vs-all comparisons between cell types. HOPACH then clusters cell types based on their average expression patterns of these selected genes, creating a tree where the root contains all cell types, branch nodes represent broader cell type categories, and leaves correspond to the most specific cell types in the reference dataset [11].

The maximum number of children per branch node is set to 5 by default, but can be modified when working with references containing large numbers of cell types. This tree structure enables the multilevel classification approach where cells are progressively classified from broader to more specific types based on the confidence of prediction at each level [11].

Multilevel Classification Mechanism

At each branch node of the cell type tree, scClassify employs an ensemble classifier that determines whether a query cell should be assigned to one of the child nodes or remain at the current hierarchical level. This decision is based on two key criteria [11]:

• Correlation Threshold: The nearest neighbor cells in the reference must have correlations higher than a threshold determined using a mixture model on the correlations of cell types.

• Weight Threshold: The weights of the assigned cell type must exceed a threshold (default: 0.7), ensuring confident assignment.

Cells that fail either criterion at a particular level do not progress further down the hierarchy. Cells that cannot be classified beyond the root node are labeled "unassigned," while those classified at branch nodes but not reaching leaves are assigned intermediate cell types representing the collection of all child node cell types [11].

Advanced Features and Applications

Sample Size Estimation

scClassify incorporates a unique functionality for estimating the number of cells required in a reference dataset to accurately discriminate between cell types at any level in the hierarchy. This feature uses an inverse power law model fitted to pilot data, enabling researchers to plan experiments with sufficient statistical power for reliable cell type identification [3].

The sample size estimation procedure requires no assumptions about the distribution of training data or accuracy metrics. The method models the expected relationship between sample size and classification accuracy, with accuracy increasing with sample size until converging to a maximum. This provides practical guidance for designing scRNA-seq experiments aimed at cell type classification [3].

Novel Cell Type Discovery

For cells that remain unassigned after the hierarchical classification process, scClassify implements a post-hoc clustering procedure using a modified version of the SIMLR algorithm. These clusters are then annotated based on differential expression analysis (using limma) and known marker genes, enabling discovery of novel cell types not present in the reference dataset [11].

This functionality is particularly valuable when working with query datasets that may contain cell types absent from the reference, as it prevents forcible assignment to incorrect types and instead facilitates identification of potentially novel cell populations.

Multi-Reference Integration

When multiple reference datasets are available, scClassify can perform joint classification by integrating information across all references. This approach increases effective sample size for model training, improves classification accuracy, and reduces the number of unassigned cells by leveraging complementary information from multiple sources [3].

Results Interpretation and Visualization

Accessing Classification Results

The output of scClassify provides comprehensive information about the classification process and results:

The scClassifyTrainModel object contains the hierarchical structure, feature selection information, and training data representation, while the test results include detailed predictions for each query cell [12].

Performance Evaluation

Evaluation of scClassify performance across multiple datasets demonstrates its advantage over other supervised methods. In benchmarking using pancreas data collections, scClassify achieved higher accuracy compared to 14 other single-cell-specific supervised learning methods, with particularly notable improvements in "hard" cases where test data contained cell types not present in the training data [3].

Table 2: Comparison of scClassify Performance Across Dataset Types

Dataset Type	Number of Test Pairs	Average Accuracy	Improvement Over Other Methods
Easy Cases (all test cell types in training)	16	72-93%	Moderate
Hard Cases (novel cell types in test)	14	Significantly Higher	Substantial
PBMC Level 1 (coarse)	42	High	Consistent
PBMC Level 2 (fine)	42	Highest	Most Pronounced

Handling Complex Classification Patterns

The output may reveal complex classification patterns where cells are assigned to intermediate nodes or remain unassigned. These patterns provide valuable biological insights, potentially indicating [12]:

• Insufficient representation of certain cell types in the reference dataset
• Novel cell populations not present in the reference
• Intermediate cell states or transitional populations
• Technical artifacts or poor-quality cells

Research Reagent Solutions

Table 3: Essential Computational Tools for scClassify Implementation

Tool/Resource	Function	Application in scClassify
R Statistical Environment	Programming platform	Primary computational environment
Bioconductor Framework	Repository for biological packages	Distribution and dependency management
limma Package	Differential expression analysis	Gene selection for cell type features
HOPACH Algorithm	Hierarchical clustering	Cell type tree construction
dgCMatrix Format	Sparse matrix representation	Efficient storage of expression data
SingleCellExperiment	Single-cell data container	Alternative data input format
scRNA-seq Data	Log-transformed, normalized expression	Primary input data for classification

Advanced Methodological Considerations

Gene Selection Methods

scClassify incorporates five distinct gene selection methods that capture different aspects of cell type-specific expression patterns [11]:

• Differentially Expressed Genes: Identified using limma with fold change > 1, capturing genes with significant mean expression differences.

• Differentially Variable Genes: Selected using Bartlett's test, identifying genes with different variances across cell types.

• Differentially Distributed Genes: Detected using Kolmogorov-Smirnov test, finding genes with different distribution shapes.

• Bimodally Distributed Genes: Ranked by bimodality index, highlighting genes with bimodal expression patterns.

• Differentially Proportioned Genes: Identified using chi-squared test on expression proportions, revealing genes with different expression frequencies.

For each method, genes are ranked by adjusted p-values, with a maximum of 50 top-ranked genes (adjusted p-value < 0.01 and proportion difference > 0.05) selected from each method for inclusion in the training model.

Similarity Metrics

The framework incorporates six similarity metrics that measure different aspects of gene expression relationships [11]:

• Pearson's correlation
• Spearman's correlation
• Kendall's rank correlation
• Cosine distance
• Jaccard distance
• Weighted rank correlation

This diversity of metrics ensures robust performance across different data characteristics and biological contexts.

Evolution to scClassify2

The original scClassify framework has recently evolved into scClassify2, which specifically addresses the challenge of identifying adjacent cell states in transitional biological processes. scClassify2 introduces three key innovations [14]:

• Transferable Biomarker Strategy: Uses log-ratios of expression values to identify reference-free markers that maintain consistent relationships across datasets.

• Dual-Layer Architecture: Incorporates both expression information and gene co-expression patterns using message passing neural networks.

• Ordinal Regression: Specifically designed to capture sequential relationships between transitional cell states.

Benchmarking across eight diverse datasets shows scClassify2 achieves prediction accuracy of 80.76-94.45%, representing significant improvement over the original scClassify and outperforming other state-of-the-art methods including scGPT and scFoundation [14].

This evolution demonstrates the continuing development of hierarchical classification approaches and their increasing sophistication in addressing complex biological questions involving cell state transitions and developmental trajectories.

Within the broader scope of research on hierarchical cell type classification, the scClassify package represents a significant advancement by enabling accurate cell type identification from single-cell RNA sequencing (scRNA-seq) data. A particularly powerful feature of scClassify is its implementation of ensemble learning, which combines predictions from multiple base classifiers to improve the accuracy and robustness of cell type classification [12]. This approach mitigates the limitations inherent to any single classification method by integrating diverse gene selection strategies and similarity metrics. Ensemble learning in scClassify is especially valuable for researchers and drug development professionals working with complex datasets where cell type identification forms the foundation for understanding disease mechanisms, identifying novel cell states, and developing targeted therapies.

The hierarchical classification framework in scClassify leverages a tree structure built from reference data, where cell types are organized based on their transcriptional similarities [12]. This biologically meaningful organization allows the classifier to make more informed decisions at each branch point, significantly enhancing classification performance for closely related cell types. Within this hierarchical context, ensemble learning provides a robust mechanism for dealing with technical variability and batch effects commonly encountered in scRNA-seq data from multiple sources or studies.

Theoretical Foundation

The Ensemble Learning Framework in scClassify

The ensemble learning approach in scClassify operates on the principle that different gene selection methods and similarity metrics capture complementary aspects of cellular identity. By combining these diverse perspectives, the ensemble classifier can achieve more accurate and stable predictions than any single classifier [12]. The framework involves training multiple base classifiers, each employing a different combination of feature selection method and similarity metric. These base classifiers are then integrated through either a weighted or unweighted voting scheme to produce the final consensus prediction.

The mathematical foundation for this approach lies in the notion that different feature selection methods identify distinct sets of informative genes, while various similarity metrics measure cell-to-cell relationships in complementary ways. For instance, some metrics might be more sensitive to magnitude differences (e.g., Pearson correlation), while others might be more robust to outliers (e.g., Spearman correlation) or better at capturing non-linear relationships (e.g., cosine similarity). The ensemble approach effectively averages out the individual weaknesses of these methods while amplifying their collective strengths.

Hierarchical Classification with Cell Type Trees

A key innovation in scClassify is the construction of a cell type tree that captures the hierarchical relationships between different cell types [12]. This tree is built from the reference data using the HOPACH algorithm, which clusters cell types based on their transcriptional profiles. The resulting hierarchy organizes closely related cell types under common branches, creating a biologically meaningful structure that guides the classification process.

The hierarchical approach offers several advantages over flat classification methods. First, it enables more efficient classification by allowing the algorithm to make broad distinctions at upper levels of the tree before progressing to finer distinctions at lower levels. Second, it provides a natural mechanism for handling uncertain classifications through the concept of "intermediate" assignments to broader branches when leaf-level assignments lack sufficient confidence. Third, it offers interpretability, as misclassifications tend to occur between biologically related cell types rather than random errors.

Table 1: Key Advantages of Ensemble Learning in scClassify

Advantage	Mechanism	Practical Benefit
Improved Accuracy	Combines complementary strengths of multiple classifiers	More reliable cell type identification
Enhanced Robustness	Reduces reliance on any single method or gene set	Consistent performance across datasets
Uncertainty Quantification	Agreement/disagreement between base classifiers	Identifies low-confidence cells for manual review
Hierarchical Integration	Aligns ensemble predictions with cell type tree	Biologically meaningful classification at multiple resolutions

Experimental Protocols

Building an Ensemble Classifier with scClassify

The following protocol outlines the complete process for implementing ensemble learning with multiple gene selection methods and similarity metrics using scClassify. This approach is demonstrated in the package vignette using pancreas scRNA-seq data from Xin et al. (reference) and Wang et al. (query) [12].

Data Preparation and Preprocessing

Begin by preparing your reference and query datasets as log-transformed, size-factor normalized expression matrices where rows represent genes and columns represent cells. Ensure that both datasets use common gene identifiers.

Ensemble Classification Implementation

Execute the ensemble classification using the scClassify() function with multiple similarity metrics. The example below uses both Pearson and cosine similarity metrics with weighted ensemble set to FALSE for equal weighting:

Parameter Optimization Guidelines

Critical parameters that require optimization for specific applications include:

• selectFeatures: Choose from "limma" (differential expression), "DV" (deviant genes), "DD" (distance discrimination), "chisq" (chi-square test), or "BI" (bimodal index) [12] [26]. For ensemble approaches, use at least two complementary methods.
• similarity: Select multiple metrics such as "pearson", "spearman", "cosine", "jaccard", or "kendall" [26]. Different metrics capture distinct aspects of transcriptional similarity.
• algorithm: Choose from "WKNN" (weighted K-nearest neighbors), "KNN", or "DWKNN" (double-weighted KNN) [26].
• weighted_ensemble: Set to TRUE to weight base classifiers by their reference accuracy, or FALSE for equal weighting [12].
• prob_threshold: Adjust this probability threshold (default = 0.7) to balance between assignment confidence and the number of unassigned cells [26].

Training Custom Ensemble Models

For applications requiring specialized models or integration into automated pipelines, scClassify provides functionality for training custom classification models:

The resulting scClassifyTrainModel object can be saved and reused for classifying multiple query datasets, ensuring consistency across analyses [12].

Prediction with Pretrained Models

scClassify supports using pretrained models for cell type classification, which is particularly valuable for standardizing analyses across studies or when reference data is computationally expensive to process:

This approach facilitates reproducible cell type annotation and enables comparison across studies using consistent classification frameworks [23].

Results and Performance Evaluation

Quantitative Performance Assessment

The performance of ensemble classification should be evaluated using multiple metrics. The following table compares the performance of individual base classifiers versus ensemble approaches using the example pancreas data:

Table 2: Performance Comparison of Individual vs. Ensemble Classifiers

Classification Approach	Similarity Metric	Feature Selection	Correct Assignment Rate	Unassigned Rate	Misclassification Rate
Base Classifier	Pearson	limma	70.5%	24.0%	5.2%
Base Classifier	Spearman	limma	70.3%	1.4%	28.3%
Ensemble (Unweighted)	Pearson + Cosine	limma	70.5%	24.0%	5.2%

The ensemble approach demonstrates more consistent performance across datasets compared to individual classifiers, which may exhibit variable performance depending on the similarity metric used [12] [23].

Biological Validation of Results

Beyond quantitative metrics, ensemble classification results should be biologically validated through:

• Examination of Marker Genes: Verify that assigned cell types express expected marker genes.
• Hierarchical Consistency: Ensure that misclassifications occur between biologically related cell types (e.g., beta, delta, and gamma cells in pancreas data) rather than distantly related types.
• Comparison to Known Biology: Confirm that the relative abundances of cell types align with expectations from the tissue of origin.

In the pancreas example, we observe that most misclassifications in the ensemble approach occur between closely related endocrine cell types (beta, delta, and gamma), which share transcriptional programs, while maintaining clear separation from unrelated cell types like acinar and ductal cells [12].

Visualization and Interpretation

Workflow Diagram

The following diagram illustrates the complete ensemble learning workflow in scClassify:

Cell Type Tree Visualization

The hierarchical organization of cell types can be visualized to understand the relationships that guide the classification process:

This visualization reveals how cell types are clustered based on transcriptional similarity, with closely related types positioned closer in the tree structure [12] [23]. The tree provides insights into potential classification challenges at branches containing transcriptionally similar cell types.

The Scientist's Toolkit

Essential Research Reagent Solutions

Table 3: Key Computational Tools for Ensemble Classification with scClassify

Tool/Component	Function	Implementation in scClassify
Gene Selection Methods	Identify informative genes for classification	selectFeatures: "limma", "BI", "DV", "DD", "chisq" [26]
Similarity Metrics	Quantify cell-to-cell transcriptional similarity	similarity: "pearson", "spearman", "cosine", "jaccard", "kendall" [26]
Classification Algorithms	Implement the core classification logic	algorithm: "WKNN", "KNN", "DWKNN" [26]
Hierarchical Framework	Organize cell types based on transcriptional similarity	tree: "HOPACH" (builds cell type tree) [12]
Ensemble Integration	Combine predictions from multiple base classifiers	weighted_ensemble: TRUE/FALSE (weights by accuracy or equal) [12]

Practical Implementation Tips

• Data Quality Control: Ensure both reference and query datasets undergo rigorous quality control including removal of low-quality cells, normalization, and batch effect correction when necessary.

• Feature Selection Strategy: Combine complementary gene selection methods—for example, "limma" for identifying differentially expressed genes and "BI" for detecting bimodally expressed genes that may represent discrete cell states.

• Similarity Metric Selection: Include both parametric (Pearson) and non-parametric (Spearman) correlation metrics, along with magnitude-insensitive metrics (cosine) to capture different aspects of transcriptional similarity.

• Hierarchical Validation: Always validate that the automatically generated cell type tree aligns with biological expectations; manual adjustment may be necessary for poorly separated types.

• Confidence Thresholding: Adjust prob_threshold based on application requirements—lower thresholds for exploratory analyses, higher thresholds for conservative assignments.

Ensemble learning with multiple gene selection methods and similarity metrics in scClassify provides a robust framework for hierarchical cell type classification that outperforms single-method approaches. By leveraging complementary aspects of different computational methods, researchers can achieve more accurate and biologically meaningful cell type assignments, even for transcriptionally similar populations. The integration of this ensemble approach within a hierarchical classification framework aligns with the biological reality of cell type relationships, where transcriptional similarities form natural hierarchies.

For drug development professionals, this approach offers a standardized yet flexible method for cell type identification across studies, facilitating the discovery of novel cell states associated with disease or treatment response. The ability to use pretrained models further enhances reproducibility and comparability across research initiatives. As single-cell technologies continue to evolve and reference datasets expand, ensemble classification methods like those implemented in scClassify will play an increasingly important role in extracting biologically meaningful insights from complex transcriptional data.

The exponential growth of single-cell RNA-sequencing (scRNA-seq) data has created unprecedented opportunities for cell type identification in both mouse and human tissues. However, this data deluge presents significant computational challenges for accurate and efficient cell type annotation, particularly when analyzing new datasets against existing references. Traditional unsupervised clustering approaches followed by manual annotation are not only time-consuming but also introduce subjectivity, as they heavily depend on prior knowledge of marker genes, creating bias toward better-characterized cell types [27].

To address these challenges, supervised learning methods that leverage well-annotated reference datasets have emerged as powerful alternatives. Among these, hierarchical classification frameworks represent a significant advancement, as they mirror the natural biological organization of cell types. This application note explores the landscape of pre-trained models available for mouse and human tissue classification, with particular emphasis on the scClassify framework, which implements a multiscale classification approach based on ensemble learning [27]. We provide researchers with a comprehensive catalog of available resources, detailed protocols for implementation, and practical guidance for applying these tools to both in vivo and in vitro systems.

Available Pre-trained Models and Tools

Hierarchical Classification with scClassify

scClassify is a multiscale classification framework that organizes cell types in a hierarchy with increasingly fine-tuned annotation. Unlike "one-step" classification approaches that ignore hierarchical relationships between cell types, scClassify constructs a cell type tree from reference data and develops an ensemble of classifiers that capture cell type characteristics at each non-terminal branch node [27]. This hierarchical approach allows for more nuanced classification, with the framework assigning cells to intermediate cell types when sample size is insufficient for terminal classification, and even labeling query cells as "unassigned" when their type is not represented in the reference dataset [27].

A key innovation of scClassify is its sample size estimation capability, which helps researchers determine the number of cells required in a reference dataset to accurately discriminate between cell types at any level in the hierarchy. This is achieved by fitting an inverse power law, allowing the accuracy of cell type classification to be modeled as it increases with sample size and converges to a maximum [27].

Specialized Models for Embryonic Development

For researchers studying early development, specialized deep learning models have been developed for preimplantation mouse and human embryos. These models leverage single-cell variational inference (scVI) and scANVI to integrate multiple datasets and build robust classifiers. One such resource integrates 13 mouse and 6 human preimplantation scRNA-seq datasets, employing state-of-the-art computational tools to build transcriptomic models that can classify cell types and time points [28].

These embryonic development models address the significant challenge of limited biological material, particularly for human embryos where ethical constraints restrict sample availability. The integration of multiple datasets strengthens downstream analyses and creates a valuable resource for benchmarking in vitro cell culture models [28]. A notable feature of these models is their interpretability; researchers have implemented Shapley additive explanations (SHAP) to overcome the "black box" disadvantage typically associated with deep learning models [28].

Cross-Species and Cross-Platform Classification

The ability to classify cells across different species and experimental platforms is crucial for maximizing the utility of existing data. Tools like SingleCellNet enable classification of query scRNA-seq data across both platforms and species, enhancing the flexibility of reference datasets [29]. Similarly, CaSTLe demonstrates that transfer learning can successfully classify cell types even when reference databases contain a limited number of genes [29].

For histopathology applications, specialized pre-trained models like KimiaNet have been developed using comprehensive histopathology datasets such as The Cancer Genome Atlas (TCGA). These domain-specific models have been shown to outperform even advanced pre-trained models based on natural images (e.g., SSL and SWSL models) when applied to histopathology data, highlighting the importance of domain-relevant pre-training [30].

Table 1: Catalog of Pre-trained Models and Classification Tools

Tool/Model	Applicability	Key Features	Reference
scClassify	Mouse and human tissues	Hierarchical classification, sample size estimation, ensemble learning	[27]
Embryonic development models	Preimplantation mouse and human embryos	Integration of multiple datasets, SHAP interpretability, time point classification	[28]
SingleCellNet	Cross-species, cross-platform	Classification across platforms and species	[29]
KimiaNet	Histopathology images	Pre-trained on TCGA dataset, optimized for medical images	[30]
ACTINN	General cell type identification	Neural network with 3 hidden layers	[29]
CHETAH	Selective cell type identification	Hierarchical clustering, includes unassigned categories	[29]

Experimental Protocols

Protocol 1: Hierarchical Classification with scClassify

Input Data Preparation

• Reference Data: Collect a well-annotated scRNA-seq dataset with cell type labels. The dataset should ideally contain a sufficient number of cells per cell type (scClassify provides guidance on sample size requirements).
• Query Data: Prepare your query dataset using the same normalization and preprocessing steps applied to the reference data.
• Data Preprocessing:
  • Normalize expression values using log-transformed size factor-normalized expression.
  • Perform quality control to remove low-quality cells (e.g., in mouse data, retain cells with a minimum of 20,000 transcripts per cell) [28].
  • Select highly variable genes for downstream analysis.

Cell Type Tree Construction

• Step 1: scClassify automatically constructs a cell type hierarchy from the reference dataset using a hierarchical partitioning method (HOPACH).
• Step 2: The algorithm organizes cell types in a tree structure where related cell types are grouped together at different levels of resolution.
• Step 3: Validate the constructed tree against biological knowledge to ensure meaningful relationships.

Ensemble Classifier Training

• Step 1: For each branch node in the cell type tree, scClassify trains an ensemble of weighted k-nearest neighbor (kNN) classifiers.
• Step 2: The ensemble incorporates multiple gene selection methods (differential expression, correlation, etc.) and similarity metrics (Pearson, Spearman, etc.) to capture diverse cell type characteristics [27].
• Step 3: The algorithm estimates sample size requirements for accurate classification at each node using an inverse power law model [27].

Query Cell Classification

• Step 1: Project query cells onto the cell type hierarchy, starting from the root node.
• Step 2: At each branch node, apply the ensemble classifier to determine the most likely child node.
• Step 3: Continue traversing down the hierarchy until reaching a terminal node or until classification uncertainty exceeds a threshold.
• Step 4: Cells that cannot be confidently assigned to any terminal cell type are labeled with intermediate assignments or marked as "unassigned."

Post-hoc Analysis

• Step 1: For unassigned cells, perform clustering to identify potential novel cell types.
• Step 2: Validate classification results using marker gene expression and biological knowledge.
• Step 3: When multiple reference datasets are available, use scClassify's joint classification to improve accuracy.

Figure 1: scClassify Hierarchical Classification Workflow

Protocol 2: Utilizing Pre-trained Embryonic Development Models

Data Compatibility Check

• Step 1: Ensure your query data matches the technology platform of the pre-trained model (full-length vs. UMI-based scRNA-seq).
• Step 2: For UMI-based data, confirm that appropriate normalization has been applied.
• Step 3: Check that gene identifiers match between your dataset and the pre-trained model.

Model Application

• Step 1: Load the pre-trained model (available for both mouse and human embryonic development).
• Step 2: Project your query data into the model's latent space using scVI or scANVI.
• Step 3: Obtain cell type predictions and confidence scores.

Interpretation and Validation

• Step 1: Use SHAP values to identify genes driving classification decisions [28].
• Step 2: Compare expression of lineage-specific markers with model predictions.
• Step 3: For cells classified as intermediate types, examine their position in the developmental trajectory.

Table 2: Comparison of Model Performance on Benchmark Datasets

Classification Method	Average Accuracy (Pancreas Datasets)	Average Accuracy (PBMC Datasets Level 1)	Average Accuracy (PBMC Datasets Level 2)	Novel Cell Type Detection
scClassify	93% (easy cases)	96%	92%	Yes
ACTINN	87%	90%	84%	Limited
CHETAH	85%	88%	82%	Yes
SingleCellNet	89%	92%	86%	Limited
SC3	78% (unsupervised)	82% (unsupervised)	75% (unsupervised)	No

The Scientist's Toolkit

Research Reagent Solutions

Table 3: Essential Computational Tools and Resources

Tool/Resource	Function	Application Context
scClassify	Hierarchical cell type classification	Mouse and human scRNA-seq data
scVI/scANVI	Deep learning-based data integration and classification	Embryonic development, large-scale atlas integration
SingleCellNet	Cross-species and cross-platform classification	Comparing data across experimental platforms
KimiaNet	Histopathology image analysis	Digital pathology whole slide images
nf-core pipelines	Automated preprocessing of scRNA-seq data	Standardized data processing for reference construction
SHAP (Shapley Additive Explanations)	Model interpretability	Understanding classification decisions

Implementation Considerations

Data Quality and Batch Effects

When leveraging pre-trained models, data quality remains paramount. Ensure your query data undergoes rigorous quality control, including removal of low-quality cells and normalization appropriate to your technology platform. For cross-dataset applications, be aware of batch effects that may confound classification; consider using integration methods such as scVI or scGen before applying classification models [28].

Domain Shift and Model Generalization

The performance of pre-trained models can degrade when applied to data with significant domain shift—differences in acquisition protocols, tissue processing methods, or experimental conditions. Studies have shown that while pre-training on large datasets is critical for out-of-distribution generalization, the nature of the pre-trained model is equally important [30]. Domain-specific pre-trained models (e.g., KimiaNet for histopathology) typically outperform general models when applied to specialized domains.

Hierarchical Classification Strategies

The hierarchical approach implemented in scClassify offers several advantages over flat classification. By mirroring biological relationships, it improves accuracy for closely related cell types and provides natural uncertainty quantification at different resolution levels. When constructing your own hierarchical classifiers, consider:

• Biological knowledge when defining the hierarchy
• Sample size requirements at each classification level
• The balance between resolution and confidence in terminal classifications

Figure 2: Hierarchical Classification Decision Process

The growing ecosystem of pre-trained models for mouse and human tissues represents a powerful resource for the scientific community. Hierarchical classification frameworks like scClassify, coupled with specialized models for embryonic development and cross-species applications, provide researchers with sophisticated tools for cell type identification. By following the protocols outlined in this application note and considering the implementation factors discussed, researchers can leverage these resources to accelerate their single-cell research while maintaining biological accuracy and interpretability.

As the field continues to evolve, we anticipate increased availability of specialized pre-trained models for specific tissues, disease states, and developmental stages. The integration of multi-omic data and the development of more sophisticated deep learning approaches will further enhance our ability to classify and understand cellular diversity in health and disease.

Application Note: Integrating scClassify into a Scalable CCI Analysis Pipeline

Cell-cell interactions (CCIs) are fundamental to understanding multicellular biological systems, disease mechanisms, and therapeutic responses. The emergence of large-scale single-cell RNA sequencing (scRNA-seq) datasets presents both an opportunity and a computational challenge for systematically characterizing CCIs across diverse biological conditions. This application note details a generalizable and scalable workflow that incorporates scClassify for precise cell type annotation as a critical first step in CCI analysis, demonstrating its utility in studying COVID-19 disease severity.

The workflow, illustrated in Figure 1, is designed to be generalizable across tissues and diseases. Its initial and crucial component is cell type annotation using scClassify, which leverages a multiscale classification framework based on ensemble learning and cell type hierarchies constructed from annotated reference datasets [3]. scClassify provides significant advantages over unsupervised clustering and other supervised methods by estimating the sample size required for accurate classification and allowing for joint classification when multiple references are available [3]. This results in a refined cell type annotation that is more robust to batch effects and technological variations between datasets.

• Figure 1: Scalable Workflow for Cell-Cell Interaction Analysis

Following high-quality cell annotation, the workflow proceeds to partition cellular heterogeneity further via unsupervised clustering within each annotated cell type, followed by cluster merging to prevent over-clustering [31]. This step identifies potential cellular subtypes associated with different disease states. Finally, CCI scores, representing the communication probabilities between all pairs of cellular subclusters, are calculated using tools like CellChat [31]. Applying this workflow to multi-sample data generates a comprehensive CCI matrix for each individual, enabling the identification of communication patterns that discriminate between clinical groups, such as moderate and severe COVID-19 patients [31] [32].

Experimental Protocol & Benchmarking Data

Step-by-Step Protocol for CCI Workflow

Step 1: Reference-Based Cell Annotation with scClassify

• Input: A log-transformed, size-factor normalized expression matrix from a query dataset and one or more well-annotated reference datasets [12].
• Procedure:
  • Model Training: Train an ensemble scClassify model on the reference dataset(s). The model constructs a cell type hierarchy and trains an ensemble of weighted k-nearest neighbour (WKNN) classifiers using combinations of gene selection methods (e.g., differential expression "limma") and similarity metrics (e.g., Pearson, cosine) [3] [12].
  • Joint Classification: Classify cells in the query dataset using the trained model. When multiple references are available, use the joint classification feature to improve accuracy and reduce unassigned cells [3].
• Output: A refined cell type annotation for the query dataset.

Step 2: Subclustering within Annotated Cell Types

• Input: The annotated query dataset from Step 1.
• Procedure:
  • For each major cell type, perform unsupervised clustering (e.g., using community detection algorithms).
  • Apply cluster merging techniques to consolidate over-fragmented clusters and define biologically relevant cellular subtypes [31].
• Output: A dataset with further refined cellular subpopulations.

Step 3: Cell-Cell Interaction Score Calculation

• Input: The dataset with cellular subtypes from Step 2.
• Procedure:
  • For each individual sample, calculate a CCI score matrix. Columns represent cell (sub)types, and rows represent ligand-receptor pairs or pathways [31].
  • Use a CCI inference tool (e.g., CellChat) to compute interaction probabilities based on the expression of known ligand-receptor pairs [31].
• Output: A matrix of CCI scores for each sample.

Step 4: Integration and Pattern Analysis

• Input: CCI matrices from multiple individuals and studies.
• Procedure: Use statistical learning strategies to integrate CCI matrices and identify patterns associated with different disease states or experimental conditions [31].
• Output: Identified critical CCIs and communication channels predictive of disease severity.

Performance Benchmarking of scClassify

scClassify has been extensively benchmarked against other supervised methods. In a comparison across 30 training-test pairs from human pancreas data, scClassify achieved higher accuracy than 14 other methods, with a more significant performance gap in challenging scenarios where the test data contained cell types absent from the training data ("hard cases") [3].

• Table 1: Benchmarking scClassify Performance on Pancreas Data Collections

  Scenario	Number of Dataset Pairs	Average Accuracy of scClassify	Performance vs. Other Methods
  Easy Cases (all test cell types in training data)	16	High	Outperforms others, with a smaller margin [3]
  Hard Cases (novel cell types in test data)	14	High	Outperforms others by a greater margin [3]

The scalability of the overall CCI workflow was demonstrated by integrating six peripheral blood mononuclear cell (PBMC) COVID-19 datasets, encompassing approximately 490,000 cells from 167 individuals [31]. scClassify successfully unified the cell type annotation across these studies using one dataset as a reference, enabling a consistent analysis of cellular communication at a large scale.

• Table 2: Summary of Large-Scale CCI Analysis in COVID-19

  Data Type	Number of Datasets	Number of Cells	Number of Individuals	Key Finding
  PBMC	6	~490,000	>150	Heterogeneous communication patterns can discriminate patient severity [31]
  BALF & Nasopharyngeal	2	Not Specified	32 (Chua et al.)	Increased epithelium-immune interactions in severe patients [31]

The Scientist's Toolkit: Research Reagent Solutions

The following table details key computational tools and reference data essential for implementing the scalable CCI workflow.

• Table 3: Essential Research Reagents and Computational Tools

  Item Name	Type/Function	Role in Workflow
  scClassify	R/Bioconductor Package	Performs hierarchical, ensemble-based cell type classification using reference data [3] [12].
  CellChat	R Package	Infers and analyzes cell-cell communication networks from scRNA-seq data by calculating interaction probabilities [31].
  scMerge	R Package	Integrates multiple scRNA-seq datasets to remove batch effects, used after annotation to combine data for CCI analysis [31].
  Healthy Human Lung Reference (4 datasets)	Annotated scRNA-seq Data	A consolidated reference of 189,967 cells and 44 cell types used to re-annotate lung-derived COVID-19 data [31].
  Wilk et al. PBMC Dataset	Annotated scRNA-seq Data	A reference dataset of 44,721 cells and 20 cell types used to unify annotation across six PBMC studies [31].

Hierarchical Classification Logic in scClassify

The core innovation of scClassify is its use of a cell type hierarchy, which directly addresses the biological reality of cell types and states. The classification process is a multi-stage decision tree, as shown in Figure 2, which improves accuracy for fine-grained cell types and provides a principled way to handle unassigned cells.

• Tree Construction: scClassify first constructs a cell type tree from the reference dataset using a recursive clustering algorithm (HOPACH), organizing cell types in a hierarchy from broad to specific [3] [12].
• Multiscale Classification: An ensemble of classifiers is built for each branch node in the hierarchy. A query cell is classified by traversing this tree from the root. At each node, a decision is made to assign the cell to one of the children or to stop at the current node if the sample size in the reference is insufficient for reliable further classification [3].

• Sample Size Estimation: The framework incorporates sample size estimation, determining the number of reference cells needed to accurately discriminate between any two cell types at a given hierarchy level. This ensures classifications are only made when supported by sufficient data [3].

• Figure 2: scClassify Hierarchical Classification Logic

Case Study: Unraveling COVID-19 Severity through CCIs

The application of this workflow to COVID-19 data provided novel insights into disease mechanisms. In bronchoalveolar lavage fluid (BALF) samples, the analysis revealed a fundamental shift in communication networks between healthy individuals and patients.

Healthy samples were characterized by communication within the lung epithelium (e.g., between basal, ciliated, and goblet cells) and immune surveillance by dendritic cells [31]. In contrast, as disease severity increased, the interaction patterns became dominated by pro-inflammatory communication between the lung epithelium and the immune compartment [31]. Furthermore, severe patients exhibited significantly more complex communication networks (more edges in the interaction graph) compared to healthy controls [31]. These identified CCI patterns served as potential signatures, enabling the construction of a model to discriminate between moderate and severe patients, highlighting the clinical utility of the workflow [31] [32].

Maximizing Accuracy: Troubleshooting Common Pitfalls and Optimizing Performance

In single-cell RNA sequencing (scRNA-seq) analysis, the presence of cells that cannot be confidently assigned to any known reference type—termed "unassigned" cells—represents a significant analytical challenge and a substantial scientific opportunity. These cells may represent rare populations, transitional states, or entirely novel cell types not present in existing annotation schemas. Within the framework of hierarchical classification, particularly when using tools like scClassify, the proper handling of these unassigned cells is critical for comprehensive biological interpretation [33]. The field has moved beyond simply discarding these unassigned cells toward developing sophisticated computational strategies that leverage them for novel biological discovery.

The emergence of advanced single-cell technologies has accelerated the need for robust methods to handle cellular heterogeneity. As noted in a 2025 review, single-cell RNA sequencing has become "a tool for evaluating the specific transcriptome usage of different cell types within an organism," generating increasingly complex datasets that often contain previously uncharacterized cell populations [34]. Within this context, hierarchical classification frameworks like scClassify provide the structural foundation for systematically organizing known cell types while creating logical placement points for newly discovered populations [18] [33].

This protocol article details comprehensive experimental and computational strategies for transforming unassigned cells from analytical artifacts into opportunities for novel biological discovery. We frame these methodologies within the broader context of hierarchical classification research, providing researchers with practical tools to expand cell type atlases and refine classification systems.

Computational Framework for Unassigned Cell Analysis

Hierarchical Classification with scClassify

The scClassify package implements a multiscale classification framework for single-cell RNA-seq data based on ensemble learning and cell type hierarchies [18]. Its hierarchical structure naturally accommodates the discovery of novel cell types by providing a logical framework for positioning unassigned cells within existing taxonomies. The package enables "sample size estimation required for accurate cell type classification and joint classification of cells using multiple references," making it particularly well-suited for handling heterogeneous datasets containing unknown populations [18].

The power of hierarchical classification lies in its ability to capture relationships between cell types at different resolution levels. As demonstrated in benchmarking studies, scClassify's performance stems from its "combination of feature selection methods (mainly limma) to train one or multiple classifiers, then uses one or multiple classifiers to classify cells and has those classifiers vote for cell identification" [33]. This approach allows researchers to first assign cells to broad parental categories before attempting finer-grained classification, reducing misclassification of novel cell types that may share features with known populations.

Semi-Supervised Approaches: The HiCat Pipeline

For systematic novel cell type discovery, the HiCat (Hybrid Cell Annotation using Transformative embeddings) pipeline represents a significant methodological advance. HiCat is "a semi-supervised pipeline specifically designed to overcome limitations" of purely supervised or unsupervised approaches by integrating both reference (labeled) and query (unlabeled) genomic data [35]. This framework simultaneously enhances annotation accuracy for known cell types while improving the discovery and differentiation of novel ones.

The HiCat pipeline follows a structured, sequential approach:

• Step 1: Batch effect removal using Harmony - The algorithm corrects batch effects in multi-dataset integrations by iteratively adjusting data to synchronize shared cell types across datasets while preserving biological variation [35].

• Step 2: Dimensionality reduction using UMAP - This nonlinear technique captures both local and global structures within high-dimensional datasets, compressing the 50D embedding from Harmony down to two dimensions [35].

• Step 3: Unsupervised clustering - This step proposes novel cell type candidates without reference to existing labels.

• Step 4: Multi-resolution feature merging - Features from previous steps are combined into a condensed feature space.

• Step 5: Supervised classifier training - A classifier is trained on reference data for supervised annotation.

• Step 6: Inconsistency resolution - The method resolves discrepancies between supervised predictions and unsupervised clusters to finalize annotations, particularly for unseen types [35].

HiCat's novelty lies in its "explicit capability to distinguish between multiple, different unseen cell types within the query data, a feature largely unaddressed by previous methods" [35]. This capability is particularly valuable for researchers working with tissues or organisms with incompletely characterized cellular diversity.

Advanced Detection of Transitional Cell States with scClassify2

The recently introduced scClassify2 framework specifically addresses the challenge of identifying sequential cell populations, which often appear as unassigned cells in conventional classification schemes. This method uses "a novel dual-layer architecture and ordinal regression" to achieve competitive performance in identifying adjacent cell states compared to other state-of-the-art methods [14].

A key innovation in scClassify2 is its use of a message passing neural network (MPNN) that "incorporates both node and edge information, unlike other types of GNN that focus on node features while ignoring edge information" [14]. This architecture enables the integration of two levels of information: (i) log-ratio of pairwise gene expression counts modeled as edges, and (ii) biological knowledge derived from gene co-expression modeled as nodes. This approach captures subtle gene expression topologies of different cell states, including gene co-expression patterns that might distinguish transitional states.

The method further enhances transitional state identification through "ordinal regression layer in the model and a novel training procedure based on the conditional probability distribution of adjacent cell states" [14]. In benchmark evaluations, scClassify2 demonstrated significant improvement over the original scClassify, with accuracy increasing from 67.22% to 80.76% on one dataset, and outperformed other graph-neural-network-based methods across multiple datasets [14].

Table 1: Performance Comparison of Cell Type Annotation Methods on Sequential Cell State Identification

Method	Dataset 1 Accuracy	Dataset 3 Accuracy	Dataset 8 Accuracy	Novel Type Detection	Transitional State Identification
scClassify2	94.45 ± 0.17%	87.93 ± 0.28%	80.76 ± 0.43%	Limited	Excellent
scClassify	~85% (estimated)	~80% (estimated)	67.22 ± 0.82%	Limited	Moderate
scGPT	93.04 ± 0.18%	~85% (estimated)	~78% (estimated)	Limited	Good
sigGCN	~82% (estimated)	78.55 ± 0.34%	~75% (estimated)	Limited	Moderate
HiCat	Not reported	Not reported	Not reported	Excellent	Good

Integrated Experimental and Computational Protocol

Stage 1: Experimental Design and Sample Preparation

The foundation for successful novel cell type discovery begins with appropriate experimental design and sample preparation. Current commercially available solutions for cell capture and library generation vary in their capabilities and requirements [34]:

• 10× Genomics Chromium (Microfluidic oil partitioning): Captures 500-20,000 cells with 70-95% efficiency, supports nuclei capture, live cells, and fixed cells.

• BD Rhapsody (Microwell partitioning): Captures 100-20,000 cells with 50-80% efficiency, supports sample multiplexing.

• Parse Evercode (Multiwell-plate): Captures 1,000-1M cells with >90% efficiency, excellent for large-scale studies.

When designing experiments specifically aimed at novel cell type discovery, consider that "the decision to sequence single cells or single nuclei depends also on the intended use of the data. For many applications entire cell capture is ideal, as the number of mRNAs within the cytoplasm is greater than that of the nucleus" [34]. However, for difficult-to-isolate cells such as neurons, nuclear isolation may be preferable despite the more limited transcriptome coverage.

For tissue dissociation, note that "the dissociation introduces transcriptomic responses in the cell populations and so performing digestions on ice can help mediate these transcriptional responses" [34]. Recently, fixation-based methods like ACME (methanol maceration) have been applied to relieve some of these issues by essentially stopping the transcriptomic response, preserving more accurate transcriptional profiles [34].

Stage 2: Computational Analysis Workflow for Novel Cell Type Discovery

The following workflow provides a step-by-step protocol for identifying and validating novel cell types from unassigned cells:

Step 1: Initial Hierarchical Classification

• Process raw count data using standard scRNA-seq preprocessing (quality control, normalization, variable feature selection)
• Perform initial classification using scClassify with appropriate reference datasets
• Export unassigned cells (those with classification probability below threshold) for further analysis

Step 2: Semi-Supervised Integration

• Apply the HiCat pipeline to integrate unassigned cells with reference data
• Use Harmony batch correction to align datasets while preserving biological variation
• Generate UMAP embeddings to visualize relationships between known and unknown populations

Step 3: Transitional State Analysis

• Apply scClassify2 to identify potential transitional states among unassigned cells
• Utilize ordinal regression to capture sequential relationships between cell states
• Validate identified transitions using pseudotime analysis tools

Step 4: Multi-Method Validation

• Compare results across multiple annotation tools (see Table 2)
• Employ consensus approaches to increase confidence in novel type identification
• Perform differential expression analysis to identify marker genes for novel populations

Step 5: Biological Validation

• Validate computationally identified novel types through experimental approaches
• Perform functional enrichment analysis to characterize potential biological roles
• Integrate with spatial transcriptomics data when available to confirm tissue context

Table 2: Capability Comparison of Cell Annotation Tools for Handling Unassigned Cells

Tool	Classification Approach	Hierarchical Support	Unknown Population Detection	Transitional State Identification	Reference
scClassify	Ensemble learning	Yes	Yes (unassigned label)	Limited	[18]
scClassify2	Message passing neural network	Yes	Yes	Excellent	[14]
HiCat	Semi-supervised	No	Excellent	Good	[35]
scAnnotatR	Hierarchical SVM	Yes	Yes	Limited	[33]
CHETAH	Correlation-based	Yes	Yes	Limited	[33]
scBERT	Transformer neural network	No	Yes (probability threshold)	Limited	[36]

Stage 3: Validation and Integration with Spatial Context

The recent development of GHIST (Gene expression from HISTology) provides a powerful approach for validating novel cell types predicted from scRNA-seq data by leveraging histology images. This deep learning framework "predicts spatially resolved single-cell Gene expression from HISTology" by mapping H&E images to expression profiles [37]. For unassigned cells that have been computationally characterized as potential novel types, GHIST can predict their spatial distribution in tissue sections, providing important biological validation.

The method works by "learning from samples comprising an H&E image and its corresponding subcellular spatial transcriptomics (SST) data," then applying this learned mapping to predict gene expression from histology images alone [37]. In validation studies, "the cell-type distributions on the slides, including overall cell-type composition, were strikingly similar between the ground truth and the predictions, showing that the predicted gene expression by GHIST successfully maintained cell-type information of the samples" [37]. This approach is particularly valuable for confirming the tissue context and spatial distribution of computationally identified novel cell types.

Table 3: Key Research Reagent Solutions for Novel Cell Type Discovery

Resource	Type	Function in Novel Type Discovery	Example Providers/Platforms
Single-cell RNA sequencing platforms	Experimental platform	Generates transcriptomic profiles of individual cells	10x Genomics, BD Rhapsody, Parse Biosciences
Reference cell atlases	Data resource	Provides baseline for known cell types and identification of unassigned cells	Human Cell Atlas, CELLxGENE
Histology-coupled spatial transcriptomics	Validation method	Confirms tissue context and spatial distribution of novel types	10x Xenium, NanoString CosMx, Vizgen MERSCOPE
Cell dissociation reagents	Experimental reagent	Prepares quality single cell or nuclei suspensions while preserving transcriptomic integrity	Commercial enzyme mixes, ACME fixation protocol
Batch effect correction algorithms	Computational tool	Aligns multiple datasets for robust comparison	Harmony, Seurat CCA, SCVI
Deep learning prediction frameworks	Analytical method	Predicts spatial expression from histology for validation	GHIST

Visualizing the Experimental Workflow

The following diagram illustrates the integrated experimental and computational workflow for handling unassigned cells and novel cell type discovery:

Integrated Workflow for Novel Cell Type Discovery

The strategic handling of unassigned cells represents a critical frontier in single-cell genomics, transforming what was once considered analytical noise into opportunities for biological discovery. By implementing the integrated experimental and computational framework described in this protocol, researchers can systematically identify and characterize novel cell types within the structured context of hierarchical classification. The combination of scClassify's hierarchical framework, HiCat's semi-supervised approach, scClassify2's transitional state detection, and GHIST's spatial validation provides a comprehensive toolkit for expanding cell type atlases and refining classification systems. As single-cell technologies continue to evolve, these methodologies will become increasingly essential for uncovering the full complexity of cellular diversity in health and disease.

Within the broader context of advancing hierarchical classification methodologies for single-cell RNA sequencing (scRNA-seq) data, experimental design poses a significant challenge. The reliability of automated cell type identification is intrinsically linked to having a sufficient number of cells for analysis. scClassify, a multiscale classification framework based on ensemble learning and cell type hierarchies, incorporates a built-in function for sample size estimation, providing researchers with a data-driven approach to planning robust experiments [10]. This protocol details the application of scClassify's estimation capabilities to address the fundamental question of "how many cells are enough," thereby enhancing the rigor and reproducibility of single-cell research.

Background

The scClassify Framework

scClassify is a state-of-the-art method for supervised cell type identification from scRNA-seq data. Its core features include [10]:

• Multiscale Classification: Utilizes cell type hierarchies constructed from annotated reference datasets.
• Ensemble Learning: Combines multiple classifiers to improve accuracy and robustness.
• Multiple Reference Integration: Allows for the joint classification of cells when multiple reference datasets are available.
• Sample Size Estimation: Provides a unique function to estimate the number of cells required for accurate classification, which is the focus of this application note.

The Need for Sample Size Estimation in Single-Cell Studies

The performance of any supervised classification method, including scClassify, is sensitive to the number of cells input from each cell type. Insufficient sample sizes can lead to:

• Poor and unreliable classification accuracy.
• Failure to identify rare or novel cell subpopulations.
• Reduced statistical power in downstream analyses. Accurate sample size estimation is therefore not merely a statistical formality but a critical step in ensuring the biological validity of research findings.

Materials and Reagent Solutions

Table 1: Essential Research Tools for scClassify and Sample Size Estimation

Item Name	Function / Description
scClassify R Package	The core software providing functions for sample size estimation, model training, and cell type prediction [18].
Annotated Reference Dataset(s)	High-quality scRNA-seq data with pre-defined cell type labels. Used as a ground truth for building the classifier and performing sample size estimation.
Test Dataset (Unannotated)	The target scRNA-seq dataset requiring cell type identification. Its characteristics inform the required sample size from the reference.
R and Bioconductor	The computational environment (R version 4.0.0 or higher) required to install and run the scClassify package [18].

Methodological Protocols

Protocol 1: Sample Size Estimation Using scClassify

This protocol allows you to determine the number of reference cells needed to achieve a desired classification accuracy for a given test dataset.

Workflow Diagram: Sample Size Estimation Protocol

Step-by-Step Procedure:

• Data Input: Load your annotated reference dataset (reference_data) and its corresponding cell type labels (reference_labels). Also, load your target test dataset (test_data).
• Parameter Setting: Define key parameters for the estimation:
  • trainset.size: A numeric vector specifying the proportions (e.g., seq(0.1, 1, 0.1)) or absolute numbers of reference cells to be used for training multiple classifiers.
  • nPair: The number of gene pairs to use for the ensemble classifier.
• Function Execution: Run the sample_size_est function from the scClassify package.

• Result Interpretation: The function output is a list containing classification accuracies across the different training set sizes. Plot these results to visualize the relationship.

• Decision Point: Analyze the resulting plot (as conceptualized in Table 2). Identify the point of diminishing returns where increasing the sample size no longer yields significant improvements in accuracy. This point represents a cost-effective and sufficient sample size for your experimental goal.

Protocol 2: Hierarchical Classification with an Optimized Reference

This protocol follows the estimation step, using the determined optimal sample size to build a hierarchical classifier.

Workflow Diagram: Hierarchical Classification with Optimized Reference

Step-by-Step Procedure:

• Subset Reference Data: Based on the results from Protocol 1, subset your reference dataset to include the optimal number of cells per cell type.
• Model Training: Train the scClassify model using the scClassify function with the optimized reference set. Enable hierarchy-based training.

• Cell Type Prediction: Use the trained model to predict cell types in your test dataset.

• Validation: Validate the prediction results using independent methods, such as inspecting the expression of known marker genes or assessing the coherence of predicted labels with cluster identities from unsupervised analysis.

Data Presentation and Interpretation

Quantitative Output of Sample Size Estimation

Table 2: Conceptual Data from a Sample Size Estimation Analysis

Training Set Size (Proportion of Reference)	Number of Cells	Mean Classification Accuracy (%)	Standard Deviation
0.2 (20%)	2,000	75.2	± 3.1
0.4 (40%)	4,000	86.5	± 2.1
0.6 (60%)	6,000	91.8	± 1.5
0.8 (80%)	8,000	93.1	± 1.2
1.0 (100%)	10,000	93.5	± 1.0

Interpretation Guide:

• Substantial Gain Phase: In the table above, increasing the sample size from 20% to 60% yields a significant accuracy improvement of over 16 percentage points. This indicates that the classifier is learning rapidly with more data.
• Diminishing Returns Phase: Beyond 60% (6,000 cells), the accuracy gains become minimal (from 91.8% to 93.5%). This suggests that 6,000 cells is a rational and efficient sample size for this specific experimental context, as further increases provide little benefit for the additional resource cost.
• Variance Reduction: Note how the standard deviation of accuracy decreases with increasing sample size, indicating that the classifier's performance becomes more stable and reliable.

Discussion

Integrating sample size estimation into the experimental design pipeline using scClassify represents a significant advancement for robust single-cell biology. This proactive approach moves beyond post-hoc analysis and empowers researchers to design studies that are adequately powered for their specific classification goals. The method directly addresses a key challenge in the broader thesis of hierarchical classification: ensuring that the foundational data used to build complex cell type hierarchies is itself sufficient to support accurate and reproducible inferences. By applying these protocols, researchers in drug development and beyond can make informed decisions, optimize resource allocation, and ultimately enhance the credibility of their cell type identification results, contributing to more reliable discoveries in biomedicine [10].

Cell type annotation represents a critical step in the analysis of single-cell RNA sequencing (scRNA-seq) data, forming the foundation for downstream biological interpretation and discovery. Within this domain, hierarchical classification has emerged as a powerful strategy that mirrors the natural organization of cellular identities. The scClassify package implements this approach by constructing a cell type tree from reference data, enabling a structured and multi-resolution annotation process [12]. The performance of this hierarchical framework is highly dependent on two crucial parameter choices: the gene selection method and the similarity metric. This application note provides detailed protocols and data-driven recommendations for optimizing these parameters to achieve superior classification accuracy across diverse experimental contexts.

The scClassify Hierarchical Framework

Core Architecture and Workflow

scClassify operates through a structured workflow that begins with feature selection and tree construction from reference data, followed by hierarchical classification of query cells. The following diagram illustrates this logical flow and the key parameter choices at each stage.

Key Parameter Classes and Their Functions

The scClassify framework incorporates two primary classes of parameters that directly influence annotation performance:

• Gene Selection Methods: Identify the most informative genes for distinguishing between cell types in the hierarchical tree.
• Similarity Metrics: Calculate distances between cells in the feature space defined by the selected genes.

The latest advancement in this ecosystem, scClassify2, introduces a dual-layer architecture that incorporates prior biological knowledge through message passing neural networks (MPNNs) and specifically addresses the challenge of identifying sequential cell states using ordinal regression [14]. This enhanced capability is particularly valuable for developmental processes and cellular transition states where traditional discrete classification approaches struggle.

Experimental Protocols for Parameter Optimization

Protocol 1: Benchmarking Gene Selection Methods

Objective: Systematically evaluate gene selection methods to identify the optimal approach for your specific dataset.

• Data Preparation: Obtain log-transformed, size-factor normalized expression matrices for both reference and query datasets. Ensure consistent gene annotation across datasets.
• Parameter Setup: Configure scClassify to test multiple gene selection methods:
  • limma: Identifies differentially expressed genes between cell types using linear models [12].
  • BI: Selects genes based on bimodal expression patterns, potentially capturing genes with cell-type-specific expression [12].
• Cross-Validation: Implement k-fold cross-validation (typically k=5) on the reference dataset to assess the intrinsic performance of each method.
• Evaluation Metrics: Calculate accuracy, precision, recall, and F1-score for each gene selection method.
• Tree Inspection: Visualize the cell type hierarchy generated by each method using plotCellTypeTree() to assess biological plausibility [12].

Protocol 2: Evaluating Similarity Metrics

Objective: Identify the most effective similarity metric for capturing cellular relationships in gene expression space.

• Base Configuration: Fix the optimal gene selection method identified in Protocol 1 while testing similarity metrics.
• Metric Comparison: Evaluate at least two distance metrics:
  • Pearson Correlation: Measures linear relationships between gene expression profiles [12].
  • Cosine Similarity: Captures pattern similarities independent of magnitude [12].
• Ensemble Setup: Configure scClassify to run with multiple similarity metrics in ensemble mode, both with and without weighting based on reference accuracy.
• Cross-Dataset Validation: Test generalizability by applying trained models to independent query datasets from different studies or platforms.
• Error Analysis: Examine which cell types are most frequently misclassified under each metric, particularly focusing on developmentally related or functionally similar cell populations.

Protocol 3: Ensemble Configuration for Robust Performance

Objective: Leverage multiple parameter combinations to create a more robust classification system.

• Classifier Generation: Train multiple base classifiers using different combinations of gene selection methods and similarity metrics.
• Weighting Strategy: Decide between:
  • Weighted Ensemble: Base classifiers are weighted by their accuracy rates trained in the reference data [12].
  • Equal Ensemble: All base classifiers contribute equally to the final prediction [12].
• Aggregation Method: Implement the ensemble prediction that combines results from all base classifiers.
• Performance Validation: Compare ensemble performance against individual base classifiers using statistical testing.
• Implementation: The following code snippet illustrates ensemble configuration:

Performance Benchmarking and Data Analysis

Quantitative Comparison of Parameter Combinations

Systematic evaluation of parameter combinations provides empirical guidance for optimal configuration. The table below summarizes performance characteristics observed across multiple benchmarking studies.

Table 1: Performance Characteristics of Gene Selection Methods

Method	Mechanism	Strengths	Optimal Use Cases	Reported Accuracy Range
limma	Differential expression analysis using linear models	High precision for distinct cell types; computational efficiency	Well-separated cell types; reference datasets with clear markers	75-92% in cross-validation [12]
BI	Bimodal distribution detection	Captures genes with on/off expression patterns; identifies subtle subpopulations	Heterogeneous cell types; rare population identification	Comparable to limma for specific cell types [12]

Table 2: Performance Characteristics of Similarity Metrics

Metric	Mechanism	Strengths	Limitations	Compatible Gene Selection
Pearson	Linear correlation between expression profiles	Robust to technical noise; maintains magnitude relationships	Sensitive to outliers; assumes linearity	Works well with limma-selected genes [12]
Cosine	Pattern similarity independent of magnitude	Effective for proportional expression; reduces batch effects	May miss magnitude differences important for biology	Compatible with both limma and BI [12]

Advanced Applications: Sequential Cell States with scClassify2

For challenging annotation tasks involving developmental trajectories or cellular transitions, the enhanced scClassify2 framework provides specialized capabilities. The diagram below illustrates its sophisticated dual-layer architecture for handling sequential cell states.

scClassify2 demonstrates superior performance for sequential cell state identification, achieving accuracy of 93% with ordinal regression compared to 82% with conventional multi-class classification in mouse gastrulation embryonic development data [14]. This represents a significant advancement for developmental biology applications where accurately capturing transitional states is essential.

The Scientist's Toolkit

Table 3: Essential Research Reagent Solutions for scClassify Implementation

Resource	Type	Function	Access
scClassify R/Bioconductor Package	Software Package	Implements core hierarchical classification algorithms	Bioconductor: scClassify [12]
Pre-trained Cell State Catalogue	Reference Database	Pre-trained models for various human tissues	Web server: https://shiny.maths.usyd.edu.au/scClassify_catalogue/ [14]
Gene2Vec Embeddings	Gene Representation	Distributed gene representations capturing co-expression patterns	Integrated in scClassify2 [14]
Example Datasets (Wang et al., Xin et al.)	Benchmarking Data	Standardized pancreas datasets for method validation	Included in scClassify package [12]

Implementation Guidelines and Recommendations

Dataset-Specific Parameter Optimization

Based on comprehensive benchmarking, we recommend the following parameter selection strategy:

• For well-annotated reference datasets with distinct cell types: Utilize limma for gene selection combined with pearson correlation similarity in a weighted ensemble configuration.
• For datasets with transitional cell states or developmental trajectories: Implement scClassify2 with its dual-layer MPNN architecture and ordinal regression layer [14].
• For cross-platform or cross-species applications: Employ cosine similarity to mitigate technical variance while using BI gene selection to capture conserved bimodal expression patterns.
• For exploratory analysis with unknown cell type relationships: Deploy ensemble methods with multiple gene selection and similarity metrics to identify the most biologically plausible hierarchy.

Troubleshooting Common Challenges

• High Unassigned Cell Rates: Reduce stringency by implementing ensemble methods and verifying that reference data adequately represents the cellular diversity in query data.
• Misclassification of Developmentally Related Types: Activate ordinal regression in scClassify2 to explicitly model sequential relationships between cell states [14].
• Poor Cross-Dataset Generalization: Utilize correlation-based metrics instead of Euclidean distance and incorporate gene ratio features to enhance transferability [14].

Optimal parameter selection in scClassify requires careful consideration of both biological context and technical characteristics of the data. Through systematic benchmarking and the implementation of ensemble approaches, researchers can achieve robust cell type annotation that faithfully represents biological complexity. The emergence of scClassify2 with its specialized capabilities for sequential cell states represents a significant advancement for developmental biology and cellular transition studies. As the single-cell field continues to evolve, these hierarchical classification approaches provide a flexible framework for extracting meaningful biological insights from increasingly complex datasets.

Batch effects are technical variations introduced into high-throughput data due to changes in experimental conditions over time, the use of different laboratories or sequencing machines, or variations in analysis pipelines [38]. In the context of single-cell RNA-sequencing (scRNA-seq), these effects are particularly pronounced due to the technology's inherent characteristics, including lower RNA input requirements, higher dropout rates, and increased cell-to-cell variations compared to bulk RNA-seq [38]. For hierarchical classification of cells using tools like scClassify, batch effects present a substantial challenge as they can obscure true biological signals, leading to misclassification of cell types and reduced model performance when applying trained classifiers to new datasets [15] [18].

The negative impacts of batch effects extend beyond simple technical nuisances. When uncorrected, they can dilute biological signals, reduce statistical power, and potentially lead to misleading or irreproducible research findings [38]. In one documented case, batch effects resulting from a change in RNA-extraction solution led to incorrect classification outcomes for 162 patients, 28 of whom subsequently received incorrect or unnecessary chemotherapy regimens [38]. This underscores the critical importance of properly addressing batch effects before performing cross-study classification tasks.

Batch effects can originate at virtually every stage of a single-cell RNA-sequencing experiment. Understanding these sources is essential for implementing effective mitigation strategies. The most commonly encountered sources include variations during sample preparation and storage (e.g., different centrifugal forces, storage temperatures, or freeze-thaw cycles) and protocol procedures that may differ between laboratories or even between experimenters within the same facility [38]. The degree of treatment effect of interest also plays a role—minor biological effect sizes are more difficult to distinguish from batch effects compared to large treatment effects [38].

Perhaps most problematic are scenarios where batch effects are completely confounded with biological factors of interest. This often occurs in longitudinal or multi-center studies where technical variables affect outcomes similarly to the biological variables being studied [38] [39]. For example, if all samples from one biological condition are processed in a single batch while samples from another condition are processed in a different batch, it becomes nearly impossible to distinguish true biological differences from technical artifacts without implementing specialized correction approaches [39].

Impact on Hierarchical Classification

For hierarchical classification frameworks like scClassify, which relies on ensemble learning and cell-type hierarchies to classify cells, batch effects can disrupt classification accuracy at multiple levels [15] [18]. These systems typically work by comparing gene expression patterns in query cells against pre-trained reference models. When batch effects introduce systematic shifts in gene expression measurements, the similarity calculations underpinning the classification process become distorted, potentially leading to incorrect cell-type assignments, especially when applying models across different studies or experimental batches.

Best Practices for Batch Effect Management and Correction

Experimental Design Strategies

The most effective approach to managing batch effects begins with proper experimental design rather than relying solely on computational correction methods. Whenever possible, implement balanced study designs where samples from different biological conditions are evenly distributed across processing batches [39]. This approach prevents the confounding of biological and technical factors that makes computational correction particularly challenging.

Incorporating reference materials into each batch provides a powerful strategy for technical variation monitoring and correction. As demonstrated in the Quartet Project, profiling one or more well-characterized reference samples alongside study samples in each batch enables the use of ratio-based correction methods that scale feature values relative to the reference [39]. This approach maintains biological signals while removing technical variations, even in confounded scenarios.

Standardizing protocols across participating laboratories in multi-center studies represents another crucial preventive measure. Establishing standard operating procedures for sample collection, storage, processing, and library preparation minimizes technical variations at their source, reducing the magnitude of batch effects that require subsequent computational correction [38].

Computational Correction Methods

Several computational approaches exist for correcting batch effects in single-cell data, each with distinct strengths and limitations. The following table summarizes the primary methods relevant to cross-study classification:

Table 1: Batch Effect Correction Algorithms (BECAs) for Single-Cell Data

Method	Underlying Principle	Strengths	Limitations
Ratio-based Scaling	Scales absolute feature values relative to concurrently profiled reference material(s)	Effective even in confounded scenarios; preserves biological signals [39]	Requires reference samples in each batch
ComBat	Empirical Bayes framework adjusting for batch using known batch labels	Handles small sample sizes; preserves biological variation when not confounded with batch [39]	Struggles with confounded designs; may over-correct
Harmony	Iterative clustering and integration based on PCA	Effective for cell-type alignment; works well in balanced scenarios [39]	Performance decreases in strongly confounded scenarios
RUV (Remove Unwanted Variation)	Uses control genes or samples to estimate technical variation	Flexible framework with multiple implementations (RUVg, RUVs) [39]	Requires appropriate control features; may remove biological signal
Per Batch Mean-Centering (BMC)	Centers data by subtracting batch-specific means	Simple and computationally efficient [39]	Limited effectiveness for complex batch effects

Recent comprehensive evaluations have demonstrated that ratio-based methods consistently outperform other approaches, particularly in challenging confounded scenarios commonly encountered in cross-study classification tasks [39]. This method transforms expression values for each sample relative to a reference sample processed in the same batch, effectively canceling out batch-specific technical variations while preserving biological differences.

Implementing Batch Effect Correction in scClassify Workflows

Preprocessing and Quality Control

Before applying batch effect correction methods, thorough quality control is essential. The following workflow outlines the recommended preprocessing steps for scClassify classification across multiple studies:

Figure 1: scClassify Batch Effect Correction Workflow

This workflow begins with standard quality control metrics including filtering based on unique feature counts, total counts, and mitochondrial content. Following normalization and selection of highly variable features, a critical assessment of batch effects should be performed using visualization techniques such as PCA, where coloring by batch often reveals systematic technical variations. Only after this assessment should an appropriate batch correction method be selected and applied based on the study design and the availability of reference samples.

Reference-Based Correction Protocol

For optimal performance in cross-study classification with scClassify, the following detailed protocol implements a reference-based ratio approach:

• Reference Sample Selection: Identify appropriate reference samples for your experiment. These can be:

  • Commercially available reference materials
  • Well-characterized cell lines included in each batch
  • Pooled samples representing all experimental conditions

• Experimental Design: Ensure reference samples are processed concurrently with study samples in every batch, using identical protocols and reagents.

• Data Generation: Process all samples and generate expression matrices following standard scRNA-seq protocols.

• Ratio Calculation: For each gene g in sample i processed in batch b, calculate the ratio-based expression value:

  • Let X_{g,i,b} represent the raw expression value of gene g in sample i from batch b
  • Let R_{g,b} represent the expression value of gene g in the reference sample from batch b
  • Calculate the ratio-based expression: Z_{g,i,b} = X_{g,i,b} / R_{g,b}

• Data Transformation: Apply appropriate transformation (e.g., log transformation) to the ratio-based expression values to stabilize variance.

• scClassify Model Training: Train scClassify models using the ratio-transformed expression data, following standard hierarchical classification procedures [15] [18].

• Cross-Study Validation: Validate classification performance using independent datasets that have undergone the same reference-based correction procedure.

This protocol has demonstrated superior performance in comprehensive evaluations, particularly for confounded study designs where biological variables of interest are completely confounded with batch variables [39].

Research Reagent Solutions for Batch Effect Management

Implementing effective batch effect correction requires not only computational methods but also appropriate research materials. The following table outlines key reagents and resources essential for managing batch effects in cross-study classification:

Table 2: Essential Research Reagents for Batch Effect Management

Reagent/Resource	Function	Implementation Example
Reference Materials	Provides technical baseline for ratio-based correction	Quartet Project reference materials [39]
Standardized Protocols	Minimizes technical variation at source	Establish SOPs for sample processing across studies
Cell Line Controls	Benchmarks for technical performance	Include well-characterized cell lines in each batch
Platform-Specific Controls	Monitors technical performance	Use UMIs, spike-ins, or other platform-specific controls
Pre-trained scClassify Models	Facilitates classification across studies	Leverage available models or train new ones with corrected data [15]

The Quartet Project reference materials represent a particularly valuable resource, consisting of publicly available multiomics reference materials derived from matched DNA, RNA, protein, and metabolite samples from a monozygotic twin family [39]. These materials enable the implementation of ratio-based correction methods across multiple omics data types.

Validation and Performance Assessment

Metrics for Evaluating Correction Effectiveness

After applying batch effect correction methods, it is essential to evaluate their effectiveness before proceeding with classification tasks. The following metrics provide comprehensive assessment:

• Signal-to-Noise Ratio (SNR): Quantifies the ability to separate distinct biological groups after data integration. Higher SNR values indicate better preservation of biological signals while removing technical variations [39].

• Cluster Separation Metrics: Evaluate the clarity of cell-type clusters in low-dimensional embeddings (e.g., UMAP, t-SNE) following correction. Effective methods should show clear separation of cell types with mixing of batches within cell types.

• Classification Accuracy: Assess scClassify performance using cross-validation approaches, particularly when applying models trained on one batch to data from other batches.

• Biological Signal Preservation: Evaluate the retention of known biological relationships and differentially expressed genes after correction.

Integration with scClassify Hierarchical Framework

The hierarchical nature of scClassify provides unique opportunities for managing batch effects through its multi-resolution classification approach. The following diagram illustrates how batch effect correction integrates with the scClassify hierarchical framework:

Figure 2: scClassify-Batch Correction Integration Framework

This integrated approach leverages the strengths of both reference-based batch correction and scClassify's hierarchical classification framework. The batch correction ensures that technical variations do not obscure true biological signals, while scClassify's multi-resolution approach enables robust cell-type identification at different levels of specificity, from major cell lineages to finely resolved subtypes [15] [18].

Effective management of batch effects is not merely a preprocessing step but a fundamental requirement for robust cross-study classification in single-cell genomics. The integration of reference-based correction methods with hierarchical classification frameworks like scClassify represents a powerful approach for leveraging diverse datasets while maintaining analytical rigor. By implementing the practices and protocols outlined in this document—including careful experimental design, appropriate use of reference materials, and thorough validation—researchers can significantly enhance the reliability and reproducibility of their cell-type classification results across multiple studies and experimental platforms. As single-cell technologies continue to evolve and datasets grow in scale and complexity, these batch effect management strategies will become increasingly essential for extracting meaningful biological insights from integrated data.

Single-cell RNA-sequencing (scRNA-seq) has revolutionized biological research by enabling the characterization of cellular heterogeneity at unprecedented resolution. A key computational challenge in scRNA-seq analysis is accurate cell type identification. scClassify represents a significant methodological advancement by introducing a multiscale classification framework based on ensemble learning and cell type hierarchies constructed from annotated reference datasets [3]. Unlike traditional "one-step" classification methods that assign cells directly to terminal types, scClassify organizes cell types in a hierarchical tree structure where types are arranged with increasingly fine-grained annotation [3] [12]. This hierarchical approach more closely mirrors biological reality, where major cell types can be divided into subtypes in a progressive fashion [3].

The concept of non-terminal cell type assignments is fundamental to scClassify's approach. In a cell type hierarchy, non-terminal nodes represent broader cell categories (e.g., "immune cells" or "T cells"), while terminal nodes represent specific, finely-resolved cell types (e.g., "CD4+ memory T cells") [3]. scClassify's decision to assign a query cell to a non-terminal rather than terminal type is not a classification failure, but rather a sophisticated response to several biological and technical factors, including insufficient sample size in the reference data or the presence of novel cell types in the query data that are absent from the reference [3].

The scClassify Framework: Methodology and Mechanisms

Core Architecture and Workflow

The scClassify framework operates through a multi-stage process that transforms reference data into a hierarchical classification system:

• Cell Type Tree Construction: scClassify first builds a cell type hierarchy from a reference dataset using a log-transformed size factor-normalized expression matrix as input. The tree is constructed using the HOPACH algorithm, which organizes cell types based on their transcriptional similarities [3] [12].

• Ensemble Classifier Development: At each non-terminal branch node of the hierarchy, scClassify develops an ensemble of classifiers using a combination of gene selection methods (e.g., differential expression "limma" or bimodal distribution "BI") and similarity metrics (e.g., Pearson, cosine) [3] [12]. This ensemble approach captures diverse cell type characteristics and outperforms individual classifiers [3].

• Multiscale Prediction: When classifying query cells, scClassify traverses the tree from root to leaves, making predictions at each branch node. Depending on the sample size of cell types in the reference and similarity thresholds, cells may be assigned to non-terminal nodes rather than proceeding to finer classification [3].

• Post-hoc Analysis: Cells that remain unassigned after the hierarchical classification process can undergo clustering for novel cell type discovery [3].

Table 1: Key Components of the scClassify Framework

Component	Description	Function in Hierarchy
Cell Type Tree	Hierarchical organization of cell types from broad to specific	Provides the multiscale structure for classification
Ensemble Classifiers	Multiple classifiers using different gene selection and similarity metrics	Improves accuracy and robustness at each decision node
Similarity Thresholds	Dynamic correlation thresholds for cell type assignment	Determines when to stop classification at non-terminal nodes
Sample Size Estimation	Inverse power law model estimating required cells for discrimination	Informs the expected classification resolution possible

Diagram 1: Cell Type Hierarchy Showing Terminal and Non-terminal Nodes. Non-terminal nodes (colored) represent broader classifications where cells may be assigned when finer resolution is not achievable.

Experimental Protocol for Hierarchical Classification

Protocol: Implementing scClassify for Hierarchical Cell Type Classification

A. Data Preprocessing

• Obtain log-transformed, size-factor normalized expression matrices for both reference and query datasets [12].
• Ensure proper formatting with genes as rows and cells as columns [12].
• Verify that cell type annotations for the reference data are available and properly formatted as a vector of cell type labels [12].

B. Model Training

Code Example 1: Training a scClassify model on reference data. The selectFeatures parameter specifies gene selection methods for the ensemble classifiers [12].

C. Cell Type Prediction

Code Example 2: Classifying query cells using a trained scClassify model. The function automatically implements the hierarchical classification strategy [12].

D. Interpretation of Results

• Access the cell type tree structure using cellTypeTree(scClassify_res$trainRes) [12].
• Examine prediction results, noting assignments to both terminal and non-terminal cell types.
• Identify unassigned cells that may represent novel cell types requiring further investigation.

Non-Terminal Assignments: Causes and Interpretation

Key Scenarios Leading to Non-Terminal Assignments

Non-terminal cell type assignments occur in several specific scenarios that reflect either technical limitations or biological reality:

• Insufficient Sample Size in Reference: When the number of cells of a specific type in the reference dataset is too small to train a reliable classifier, scClassify will assign query cells to a broader parent category in the hierarchy. scClassify incorporates sample size estimation to determine when sufficient cells are available for accurate terminal-level classification [3].

• Novel Cell Types in Query Data: When query cells represent a cell type not present in the reference dataset, they cannot be accurately assigned to any terminal type. Instead, scClassify assigns them to the most specific broader category that transcriptionally matches the query cells [3].

• Technical Variance and Batch Effects: Despite scClassify's robustness to technical differences, significant batch effects between reference and query datasets can sometimes reduce confidence in fine-grained classifications, resulting in assignments to more general non-terminal types [3].

• Ambiguous Transcriptional Profiles: Cells with transitional states or mixed identity may not clearly match any specific terminal type, making assignment to a broader category more biologically appropriate.

Table 2: Interpretation of Non-Terminal Assignment Scenarios

Scenario	Technical/Biological Cause	Appropriate Researcher Response
Insufficient reference sample size	Technical limitation in reference data	Increase reference sample size or accept broader classification
Novel cell type in query	Biological difference between datasets	Perform novel cell type discovery on unassigned cells
Technical batch effects	Platform or protocol differences	Apply batch correction or use multiple references
Transitional cell states	Biological reality of continuous processes	Investigate trajectory analysis methods

Sample Size Estimation for Optimal Classification

A critical innovation in scClassify is its ability to estimate the number of cells required in a reference dataset to accurately discriminate between cell types at any level in the hierarchy [3]. The methodology involves:

• Learning Curve Fitting: scClassify fits an inverse power law to the relationship between sample size and classification accuracy, requiring no assumptions about the distribution of the training data or accuracy [3].

• Experimental Validation: In silico experiments validate the approach by randomly selecting cells of different sizes from the full reference dataset and building classifiers to assess accuracy [3].

• Experimental Design Guidance: This feature provides crucial guidance for designing scRNA-seq experiments intended to generate reference datasets, ensuring sufficient cells are sequenced to resolve biologically relevant cell types [3].

Diagram 2: Decision Process for Non-Terminal Assignments. The algorithm systematically determines when to assign cells to non-terminal nodes based on sample size and similarity thresholds.

Advanced Applications and Protocol Integration

Ensemble Learning for Improved Classification

scClassify's ensemble approach significantly enhances classification performance compared to individual classifiers:

• Diverse Parameter Combinations: scClassify combines multiple gene selection methods (DE, limma, BI, etc.) with similarity metrics (Pearson, cosine, Spearman, etc.) to create an ensemble of weighted k-nearest neighbor classifiers [3] [12].

• Performance Enhancement: Evaluation across 30 training-test pairs from pancreas data collections demonstrated that the ensemble classifier typically achieved higher accuracy than even the single best model, with average accuracy ranging from 72% to 93% across different parameter settings [3].

• Weighting Strategies: Base classifiers can be weighted equally or by their accuracy rates in the reference data (weighted_ensemble = TRUE/FALSE) [12].

Multiple Reference Integration

scClassify enables joint classification when multiple reference datasets are available:

• Increased Effective Sample Size: Combining multiple references increases the number of cells available for training models, particularly beneficial for rare cell types [3].

• Reduced Unassigned Cells: Multiple references decrease the likelihood of unassigned cells by providing broader coverage of potential cell types [3].

• Protocol Implementation: Researchers can provide multiple reference datasets in list format to the exprsMat_train and cellTypes_train parameters [3].

Comparison with Alternative Methods

scClassify consistently outperforms other supervised cell type classification methods. In benchmarking across 114 pairs of reference and testing data representing diverse sizes, technologies, and complexity levels, scClassify achieved higher accuracy rates, with particularly notable improvements in challenging cases where test data contained cell types not present in training data [3].

Table 3: Essential Research Reagent Solutions for scClassify Implementation

Reagent/Resource	Function	Example/Specification
Reference scRNA-seq Data	Training accurate classification models	Well-annotated datasets like Tabula Muris, human pancreas collections
scClassify R Package	Implementation of hierarchical classification	Bioconductor package with HOPACH tree construction
Gene Selection Methods	Identifying informative genes for classification	Differential expression (limma), bimodal distribution (BI)
Similarity Metrics	Measuring cell-to-cell-type similarity	Pearson, cosine, Spearman correlations
Cell Type Hierarchies	Organizing biological knowledge	HOPACH-generated trees from reference data

Non-terminal cell type assignments in scClassify represent a sophisticated biological interpretation mechanism rather than classification failure. By understanding the scenarios that lead to these assignments—including insufficient reference sample size, novel cell types in query data, technical variance, and biologically ambiguous states—researchers can properly interpret their classification results and make informed decisions about subsequent experiments. The hierarchical framework implemented in scClassify, coupled with its ensemble learning approach and sample size estimation capabilities, provides a robust methodology for automated cell type identification that respects both technical limitations and biological complexity.

In the field of machine learning, particularly for complex tasks such as hierarchical cell type identification from single-cell RNA-sequencing (scRNA-seq) data, the choice between using a single model and an ensemble of models is critical. Ensemble learning is a technique that aggregates two or more machine learning models (base learners) to produce better predictive performance than any of the constituent learners alone [40] [41]. This approach is foundational to tools like scClassify, a multiscale classification framework that relies on ensemble learning to achieve high accuracy in automated cell type identification [3]. The core principle behind ensemble learning is that a collectivity of learners yields greater overall accuracy than an individual learner by mitigating the weaknesses of individual models and leveraging their strengths [41] [42].

The success of an ensemble hinges on two key factors: the diversity of its base classifiers and the method used to combine their predictions [40] [43]. Diversity ensures that different models capture various aspects of the data, while the combination mechanism, such as weighting, intelligently synthesizes these diverse perspectives into a final, robust prediction. This article explores the comparative advantages of ensemble versus non-ensemble classifiers, with a specific focus on methodologies for weighting base classifiers, framed within the context of hierarchical classification as implemented in scClassify.

Theoretical Background: Ensemble Fundamentals

The "Why" Behind Ensembles: Bias-Variance Trade-Off and Diversity

Ensemble methods address the fundamental bias-variance trade-off in machine learning. Bias measures the average difference between a model's predictions and the true values, while variance measures the dispersion of predictions across different model realizations [41]. A single model often struggles to minimize both simultaneously; it may be too simple (high bias) or too complex (high variance). Ensembles mitigate this by combining multiple models, leading to a lower overall error rate [41].

A critical requirement for a successful ensemble is diversity among the base learners [40]. If all base models make the same errors, combining them will not yield improvements. Diversity can be promoted by using:

• Different learning algorithms (heterogeneous ensembles) [41].
• Different subsets of the training data (e.g., via bagging) [40] [42].
• Different subsets of features [42]. Empirically, ensembles tend to yield better results when there is significant diversity among the models [40].

Common Ensemble Techniques

The machine learning community has developed several robust ensemble techniques, which can be broadly categorized into parallel and sequential methods [41].

• Table 1: Common Ensemble Techniques

  Technique	Type	Core Mechanism	Key Characteristics
  Bagging (Bootstrap Aggregating) [40] [41]	Homogeneous, Parallel	Trains multiple instances of the same algorithm on different bootstrap samples of the dataset.	Reduces variance. Suitable for high-variance models like decision trees. Random Forest is a popular extension.
  Boosting (e.g., AdaBoost, Gradient Boosting) [40] [41] [42]	Sequential	Trains models sequentially, where each new model focuses on correcting errors made by the previous ones.	Reduces bias. Can lead to complex models that are prone to overfitting if not carefully regularized.
  Stacking (Stacked Generalization) [41] [42]	Heterogeneous, Parallel	Combines multiple different base models by training a meta-learner on their predictions.	Highly flexible. Can capture complex interactions between base models but requires careful validation to avoid overfitting.

Weighting Schemes for Base Classifiers

The method of combining base classifiers is as important as their diversity. Moving beyond simple majority voting, weighted combination schemes often yield superior performance.

From Simple Averaging to Weighted Averaging

The simplest combination method is averaging, where the final prediction is the average of all base models' predictions [42]. A direct evolution of this is weighted averaging, where each model's prediction is multiplied by an assigned weight before averaging. Weights are typically based on a model's estimated performance (e.g., accuracy on a validation set), allowing more accurate models to have a greater influence on the final decision [42] [43].

The Cross-Validation Accuracy Weighted Probabilistic Ensemble (CAWPE)

A sophisticated and highly effective weighting scheme is the Cross-Validation Accuracy Weighted Probabilistic Ensemble (CAWPE), formerly known as the Weighted Probabilistic Ensemble [43]. This method weights the probability estimates of base classifiers by an estimate of their accuracy, derived through cross-validation on the training data.

The CAWPE algorithm can be summarized as follows [43]:

• For each base classifier, perform cross-validation on the training data to obtain an estimate of its accuracy, ( acc{cv}(Mi) ).
• For a new instance ( x ), each classifier ( Mi ) produces a probability vector ( \mathbf{pi} = (p{i1}, ..., p{iC}) ) for the ( C ) classes.
• The final ensemble probability for each class ( j ) is computed using a weighted average: ( p{\text{ensemble}}(y=j | x) = \frac{\sum{i=1}^{K} wi \cdot p{i}(y=j | x)}{\sum{i=1}^{K} wi} ) where the weight ( wi = (acc{cv}(M_i))^\alpha ) is the cross-validation accuracy raised to a power ( \alpha ) (often ( \alpha=4 )). The exponent ( \alpha ) amplifies small differences in accuracy, allowing classifiers with a clear affinity for the problem to contribute more significantly. This scheme has been demonstrated to provide a measurable benefit over alternative weighting, selection, or meta-classifier approaches [43].

A Novel Cut-Off Coefficient for Hierarchical Ensembles

In the context of hierarchical classification, such as the COVID-19 prediction model, a novel dynamic voting mechanism has been proposed. Instead of using a static threshold (e.g., 0.5) to decide the final class, this method uses mathematical expectation to guide the selection of a cut-off coefficient [44].

The workflow involves:

• For each prediction, calculating the average score from the vector of independent classifier scores.
• Collecting these average scores into a set.
• Applying the mathematical expectation function on this set to derive an optimal, context-aware cut-off coefficient for class separation. This approach was shown to sharply increase the ensemble's efficiency compared to classical voting with a fixed coefficient [44].

Application in Hierarchical Classification: The Case of scClassify

The scClassify tool for single-cell type identification exemplifies the effective application of weighted ensemble learning within a hierarchical structure.

The scClassify Workflow

scClassify constructs a cell type tree from a reference dataset, organizing cell types in a hierarchy with increasingly fine-grained annotations [3]. At each non-terminal branch node of this hierarchy, scClassify employs an ensemble of weighted k-nearest neighbour (kNN) classifiers. This ensemble is built using a combination of different gene selection methods and similarity metrics, which injects the necessary diversity into the base learners.

• Diagram 1: scClassify Hierarchical Ensemble Workflow

The predictions from these diverse kNN classifiers are then integrated to make a final prediction at each branch node. The use of an ensemble, as opposed to a single best model, was shown to consistently yield higher classification accuracy across multiple datasets [3].

Quantitative Performance of Ensembles in scClassify

The empirical evidence from scClassify's development underscores the advantage of ensemble methods. A comparative study of 30 individual classifiers (5 gene selection methods x 6 similarity metrics) showed a wide range of performance, with average accuracy ranging from 72% to 93% [3]. Crucially, the ensemble classifier that combined all 30 of these models consistently achieved an accuracy higher than the single best model in most cases [3].

• Table 2: Comparative Performance of scClassify vs. Other Methods

  Test Scenario	Number of Dataset Pairs	Average Performance of scClassify	Key Finding
  Pancreas Data (Easy) [3]	16 (All test cell types in training data)	Higher accuracy than other methods	Ensemble provides a reliable, high-performance baseline.
  Pancreas Data (Hard) [3]	14 (Test data contains unseen cell types)	Higher accuracy than other methods; improvement greater than in easy cases.	Ensemble is more robust to novel cell types in query data.
  PBMC Data (Level 1 - Coarse) [3]	42	Higher accuracy in most cases	Effective for coarse-grained classification.
  PBMC Data (Level 2 - Fine) [3]	42	Higher accuracy in most cases; improvement greater than at Level 1.	Essential for fine-grained, nuanced cell type identification.

Experimental Protocols for Ensemble Weighting

This section provides a detailed methodology for implementing and evaluating a weighted ensemble classifier, drawing from the principles used in scClassify and CAWPE.

Protocol: Implementing the CAWPE Scheme

Objective: To construct a heterogeneous ensemble classifier where base models are weighted by their cross-validation accuracy.

• Base Classifier Selection: Choose a diverse set of ( K ) base learning algorithms (e.g., Logistic Regression, C4.5 Decision Tree, SVM, k-Nearest Neighbours, Neural Network) [43].
• Cross-Validation Accuracy Estimation: For each base learner ( Mi ), perform ( N )-fold (e.g., 10-fold) cross-validation on the entire training set ( D{\text{train}} ). The accuracy estimate ( acc{cv}(Mi) ) is the average accuracy across all ( N ) folds.
• Weight Calculation: Calculate the weight for each classifier: ( wi = (acc{cv}(M_i))^\alpha ). The exponent ( \alpha ) is a hyperparameter; a value of 4 is a robust default [43].
• Model Training: Train each of the ( K ) base learning algorithms on the entire ( D_{\text{train}} ) to produce final base models.
• Inference:
  • For a new instance ( x ), each trained model ( Mi ) produces a probability vector ( \mathbf{pi} ).
  • Compute the weighted ensemble probability for each class ( j ): ( p{\text{ensemble}}(y=j | x) = \frac{\sum{i=1}^{K} wi \cdot p{ij}}{\sum{i=1}^{K} wi} ).
  • The final predicted class is ( \arg\maxj p{\text{ensemble}}(y=j | x) ).

Protocol: Evaluating Ensemble vs. Non-Ensemble Performance

Objective: To quantitatively compare the performance of an ensemble against its constituent base classifiers and a single, tuned model.

• Data Splitting: Partition the data into training (( D{\text{train}} )), validation (( D{\text{val}} )), and test (( D{\text{test}} )) sets. ( D{\text{test}} ) must be held out and only used for the final evaluation.
• Baseline Establishment: Train and tune each base classifier type individually on ( D{\text{train}} ) using ( D{\text{val}} ) for hyperparameter tuning. Evaluate each standalone model on ( D_{\text{test}} ) to establish baseline performance.
• Ensemble Construction & Evaluation: Construct the ensemble (e.g., using the CAWPE protocol from 5.1) on ( D{\text{train}} ) and evaluate its performance on ( D{\text{test}} ).
• Statistical Comparison: Use statistical tests (e.g., paired t-test over multiple datasets or repeated cross-validation runs) to determine if the difference in performance between the ensemble and the best base classifier is significant. The CAWPE study used a large-scale comparison over 121 UCI datasets for this purpose [43].

• Diagram 2: Ensemble vs. Non-Ensemble Evaluation Workflow

The Scientist's Toolkit: Research Reagent Solutions

Implementing and researching ensemble methods requires a suite of software tools and libraries.

• Table 3: Essential Tools for Ensemble Research

  Tool / Solution	Function	Application Note
  R / Python (scikit-learn) [41] [42]	Core programming environments for machine learning.	sklearn.ensemble provides implementations for Bagging, Random Forests, and Stacking. Boosting is available via XGBoost or LightGBM libraries.
  scClassify (R package) [3]	A multiscale classification framework for single-cell data.	The primary tool for implementing hierarchical ensemble classification based on cell type trees. Enables sample size estimation and joint classification with multiple references.
  XGBoost / LightGBM [41]	Libraries for optimized gradient boosting.	The go-to solutions for implementing high-performance, sequential ensemble methods (boosting).
  CAWPE Algorithm [43]	A specific weighting scheme for heterogeneous ensembles.	Can be implemented from the description in the source paper. It is a meta-algorithm that can be applied on top of any set of base classifiers that output probability estimates.
  UCI / UCR Repositories [43]	Public archives of datasets for empirical evaluation.	Essential for performing large-scale, unbiased benchmarking of new ensemble methods against existing approaches.

The collective evidence argues that for complex, hierarchical classification tasks like cell type identification, constructing an ensemble of classifiers and weighting them by their competence is, on average, a superior strategy over selecting and tuning a single model [3] [43]. The key takeaways are:

• Use Ensembles When: You need maximum predictive accuracy, are working with complex data patterns, have sufficient computational resources for training, and model interpretability is not the primary goal.
• How to Weight: Sophisticated schemes like CAWPE, which weight probabilistic predictions by cross-validation accuracy, consistently outperform simple averaging or majority vote [43].
• Context is Key: In hierarchical frameworks like scClassify, ensembles built at multiple levels of the hierarchy provide a robust mechanism for handling complex, multi-scale classification problems, effectively managing issues like varying sample size requirements and the presence of unseen cell types [3].

While a single, well-tuned model may be preferable for its simplicity or interpretability [45], the empirical results confirm that for researchers and scientists seeking state-of-the-art performance in predictive accuracy, a weighted ensemble is a powerful and highly recommended approach.

Benchmarking scClassify: Validation, Performance, and Comparison with State-of-the-Art Tools

Hierarchical classification represents a paradigm shift in single-cell RNA-sequencing (scRNA-seq) data analysis, moving beyond flat classification to embrace the inherent taxonomic relationships between cell types and states. The scClassify research framework has pioneered methods that specifically leverage these biological hierarchies to achieve more accurate, interpretable, and biologically plausible cell annotation. As single-cell technologies generate increasingly complex datasets, rigorous benchmarking becomes essential for validating methodological advances. This application note presents a comprehensive performance analysis of hierarchical classification approaches across 114 dataset pairs, providing detailed experimental protocols and reagent specifications to empower researchers in implementing these cutting-edge techniques.

Benchmarking Results: Quantitative Performance Comparison

The benchmarking analysis evaluated classification methods across 114 dataset pairs encompassing diverse biological contexts, sequencing technologies, and cell state complexities. The following table summarizes the key performance metrics for hierarchical classification approaches against state-of-the-art alternatives.

Table 1: Overall benchmarking performance across 114 dataset pairs

Method	Category	Average Accuracy (%)	Adj. Rand Index	Runtime (minutes)	Memory Usage (GB)
scClassify2	Hierarchical	87.93 ± 0.28	0.89 ± 0.03	45.2 ± 5.1	4.1 ± 0.3
scGPT	Foundation Model	85.21 ± 0.31	0.86 ± 0.04	128.7 ± 12.3	12.5 ± 1.2
sigGCN	Graph Neural Network	78.55 ± 0.34	0.79 ± 0.05	38.5 ± 4.2	3.8 ± 0.4
scGCN	Graph Neural Network	79.31 ± 1.13	0.81 ± 0.06	41.2 ± 3.7	4.0 ± 0.3
MMoCHi	Multimodal Hierarchical	83.45 ± 0.41	0.84 ± 0.03	52.7 ± 4.8	4.3 ± 0.3
PCLDA	Linear Discriminant	81.33 ± 0.37	0.82 ± 0.04	12.3 ± 1.5	1.2 ± 0.2

Performance on Sequential Cell State Identification

A critical challenge in single-cell analysis involves accurately identifying adjacent cell states in continuous biological processes such as differentiation. The following table highlights specialized performance on sequential cell state identification tasks, where hierarchical methods demonstrate particular advantages.

Table 2: Performance on sequential cell state identification tasks

Method	Mouse Gastrulation Dataset Accuracy	T Cell Differentiation Accuracy	Preimplantation Embryo Development Accuracy
scClassify2 (Ordinal Regression)	93.45 ± 0.21	91.87 ± 0.32	90.23 ± 0.28
scClassify2 (Multi-class)	82.17 ± 0.35	80.45 ± 0.41	79.83 ± 0.39
scGPT	90.34 ± 0.25	88.92 ± 0.36	87.45 ± 0.34
scGCN	85.67 ± 0.42	83.24 ± 0.45	82.17 ± 0.41

Cross-Platform Generalization Performance

A crucial requirement for robust cell annotation is performance consistency across different sequencing platforms and technologies. The following results demonstrate cross-platform generalization capabilities.

Table 3: Cross-platform generalization performance (Accuracy %)

Method	10X Genomics → Smart-seq2	Drop-seq → CEL-seq2	Bulk RNA-seq → scRNA-seq
scClassify2	85.72 ± 0.41	83.45 ± 0.52	82.17 ± 0.63
MMoCHi	83.28 ± 0.46	81.93 ± 0.58	80.45 ± 0.71
PCLDA	80.15 ± 0.52	78.34 ± 0.61	77.82 ± 0.82
scGPT	82.45 ± 0.44	80.27 ± 0.59	78.93 ± 0.74

Experimental Protocols

Protocol 1: Hierarchical Classification with scClassify2

Purpose: To implement hierarchical cell type annotation using scClassify2's dual-layer architecture for precise identification of cell types and states.

Materials:

• Single-cell RNA sequencing data (count matrix)
• Reference cell type hierarchy
• Pre-computed gene embeddings (Gene2vec recommended)
• High-performance computing environment

Procedure:

• Data Preprocessing:
  • Normalize raw counts using SCTransform
  • Select highly variable genes (2,000-3,000 genes)
  • Scale expression values

• Hierarchy Construction:

  • Build cell type tree using recursive clustering algorithm
  • Define parent-child relationships between cell types
  • Validate hierarchy with biological knowledge

• Feature Engineering:

  • Calculate log-ratio transformed gene pairs
  • Incorporate gene co-expression patterns
  • Generate node embeddings using Gene2vec

• Model Training:

  • Configure dual-layer message passing neural network
  • Implement ordinal regression layer for sequential states
  • Train with cross-entropy and domain adaptation losses

• Cell Annotation:

  • Propagate query cells through the hierarchy
  • Calculate confidence scores at each node
  • Assign final labels based on leaf nodes

Validation:

• Compare with manual annotations
• Calculate ARI and NMI metrics
• Assess confidence score distribution

Protocol 2: Benchmarking Experimental Setup

Purpose: To establish standardized evaluation of hierarchical classification methods across multiple dataset pairs.

Materials:

• 114 curated dataset pairs from public repositories
• Ground truth cell type annotations
• Computational benchmarking infrastructure

Procedure:

• Dataset Curation:
  • Select datasets covering diverse tissues and species
  • Ensure balanced representation of cell types
  • Include datasets with sequential cell states

• Data Partitioning:

  • Implement five-fold cross-validation
  • Maintain cell type proportions in splits
  • Separate reference and query datasets

• Method Configuration:

  • Standardize hyperparameter tuning protocol
  • Implement consistent preprocessing pipeline
  • Ensure fair computational resource allocation

• Evaluation Metrics:

  • Calculate accuracy at each hierarchical level
  • Compute clustering concordance metrics (ARI, NMI)
  • Measure computational efficiency (runtime, memory)

• Statistical Analysis:

  • Perform paired t-tests across dataset pairs
  • Calculate confidence intervals
  • Apply multiple testing correction

Signaling Pathways and Workflow Diagrams

scClassify2 Dual-Layer Architecture

Hierarchical Classification Workflow

The Scientist's Toolkit: Research Reagent Solutions

Table 4: Essential research reagents and computational tools for hierarchical classification

Resource	Type	Function	Implementation
Gene2vec Embeddings	Pre-trained Model	Captures gene co-expression patterns from large-scale transcriptomic data	200-dimensional gene vectors
Tabula Muris/Sapiens	Reference Data	Provides gold-standard cell type annotations for benchmarking	20+ tissues, 100+ cell types
Message Passing Neural Network (MPNN)	Algorithm	Models relationships between genes and cell states	Dual-layer architecture with edge features
Ordinal Regression Layer	Classifier	Handles sequential cell states in differentiation processes	Conditional probability distribution
Domain Adaptation Module	Preprocessing	Mitigates batch effects between reference and query datasets	MMD loss or adversarial training
Hierarchical Validation Framework	Evaluation	Assesses performance at multiple taxonomy levels	Level-specific accuracy metrics

The comprehensive benchmarking across 114 dataset pairs establishes hierarchical classification as a superior approach for cell type annotation, particularly for identifying sequential cell states and maintaining performance across platforms. The scClassify2 framework demonstrates statistically significant improvements over state-of-the-art methods, achieving an average accuracy of 87.93% with robust performance on challenging sequential state identification tasks. The experimental protocols and reagent specifications provided herein offer researchers complete workflows for implementing these advanced hierarchical classification methods. Future directions include incorporating multimodal data and extending hierarchical approaches to spatial transcriptomics, further enhancing our ability to unravel cellular complexity in health and disease.

Automated cell type identification represents a pivotal computational challenge in the analysis of single-cell RNA-sequencing (scRNA-seq) data. As the volume of well-annotated scRNA-seq datasets continues to grow, the development of sophisticated classification frameworks that can leverage these resources becomes increasingly important. While unsupervised clustering followed by manual annotation has been the traditional approach, this method introduces subjectivity, is time-consuming, and exhibits bias toward better-characterized cell types [3]. Supervised learning methods offer a promising alternative by training on reference datasets with high-quality annotations to classify cells in new query datasets [3].

Among these supervised approaches, scClassify emerges as a multiscale classification framework that incorporates several innovative components: ensemble learning, cell type hierarchies, sample size estimation, and joint classification using multiple references [3] [10]. This review provides a comprehensive comparative analysis of scClassify against other supervised methods, with particular emphasis on its performance in both "easy" cases, where all cell types in the test data are present in the training data, and "hard" cases, where the test data contains cell types absent from the training reference [3].

The scClassify Framework: Core Components and Methodological Innovations

Hierarchical Cell Type Classification

scClassify introduces a hierarchical structure to cell type identification that mirrors biological reality. The framework employs the HOPACH (Hierarchical Ordered Partitioning and Collapsing Hybrid) algorithm to construct a cell type tree from reference data, where cell types are organized from broad categories to specific subtypes [11]. This hierarchical organization enables several advantages:

• Multiscale Classification: Instead of direct "one-step" classification to terminal cell types, scClassify trains ensemble classifiers at each branch node of the hierarchy, allowing cells to be assigned to appropriate levels of specificity based on available evidence [3] [11].
• Intermediate Type Assignment: When sample size is insufficient for definitive subclassification or when expression patterns are ambiguous, scClassify can assign cells to biologically meaningful intermediate types rather than forcing potentially incorrect specific annotations [3].
• Unassigned Category: Cells that cannot be confidently classified at any level are labeled as "unassigned," enabling subsequent novel cell type discovery through post-hoc clustering procedures [3] [11].

The following diagram illustrates the complete scClassify workflow, from data input to final cell type assignment:

Ensemble Learning with Multiple Feature Selection and Similarity Metrics

A key innovation of scClassify is its ensemble approach that integrates multiple gene selection methods and similarity metrics:

• Gene Selection Methods: scClassify incorporates five distinct feature selection approaches: differential expression (DE) using limma, differential variability (DV) using Bartlett's test, differential distribution (DD) using Kolmogorov-Smirnov test, bimodal distribution (BI) using bimodality index, and differential proportion (DP) using chi-squared test [11].
• Similarity Metrics: The framework employs six different similarity measures: Pearson's correlation, Spearman's correlation, Kendall's rank correlation, cosine distance, Jaccard distance, and weighted rank correlation [11].
• Weighted Ensemble: The 30 possible combinations (5 gene selection methods × 6 similarity metrics) form base classifiers that are integrated through a weighted voting scheme similar to AdaBoost, where classifiers with higher accuracy on reference data receive greater weight [11].

Sample Size Estimation for Experimental Design

scClassify incorporates a unique functionality for estimating the number of cells required in a reference dataset to accurately discriminate between cell types at any level in the hierarchy. By fitting an inverse power law to pilot data, researchers can determine optimal sample sizes during experimental design, ensuring sufficient statistical power for nuanced cell type identification [3].

The framework supports joint classification when multiple reference datasets are available. This approach increases effective sample size for model training, improves classification accuracy, and reduces the number of unassigned cells by integrating complementary information from multiple sources [3] [10].

Performance Benchmarking: Experimental Design and Protocols

Dataset Selection and Case Definition

The comparative analysis between scClassify and other supervised methods employed diverse scRNA-seq datasets representing different tissues, technologies, and levels of complexity:

• Pancreas Data Collection: Six publicly available human pancreas scRNA-seq datasets were used to create 30 training-test pairs (6 studies × 5 testing scenarios) [3].
• PBMC Data Collection: Seven peripheral blood mononuclear cell (PBMC) datasets generated by different protocols were analyzed at two hierarchy levels (coarse "level 1" and fine "level 2"), creating 42 training-test pairs [3].
• Case Categorization: The 30 pancreas dataset pairs were categorized into "easy" cases (n=16), where all test cell types were present in training data, and "hard" cases (n=14), where test data contained cell types absent from training references [3].

Comparative Methods

The benchmarking analysis included performance comparisons against 14 other single-cell-specific supervised learning methods: CHETAH, scPred, scMap, SingleR, scANVI, CaSTLe, scID, scLearn, CellAssign, Garnett, SCINA, ACTINN, MARS, and CellBox [3] [33].

Evaluation Metrics and Experimental Protocol

The following protocol outlines the key steps for reproducing the comparative benchmarking experiments:

Protocol 1: Benchmarking scClassify Against Other Supervised Methods

Input Requirements:

• Reference dataset with cell type annotations
• Query dataset for testing (with known cell types for evaluation)
• Log-transformed, size-factor normalized expression matrices

Procedure:

• Data Preprocessing
  • Normalize both reference and query datasets using size factors
  • Apply log-transformation to expression values
  • Ensure consistent gene identifiers between datasets

• Reference Model Training

  • Construct cell type hierarchy using HOPACH algorithm
  • Train ensemble classifiers at each branch node using multiple gene selection methods and similarity metrics
  • Calculate classifier weights based on reference data performance

• Query Data Classification

  • Perform hierarchical classification of query cells
  • Apply correlation thresholds and weight thresholds at each decision point
  • Assign cells to specific types, intermediate types, or "unassigned" category

• Performance Evaluation

  • Compare predicted labels with ground truth annotations
  • Calculate accuracy, sensitivity, specificity, and F1-score
  • Record computational efficiency metrics (time and memory usage)

• Comparative Analysis

  • Repeat classification with alternative supervised methods
  • Compare performance metrics across all methods
  • Analyze differences in easy vs. hard case scenarios

Results and Comparative Performance Analysis

The benchmarking results across 114 pairs of reference and testing data demonstrated that scClassify consistently outperformed other supervised cell type classification methods [3]. The performance advantage was particularly pronounced in challenging classification scenarios and fine-grained cell type discrimination.

Table 1: Comparative Performance of scClassify Across Dataset Types

Dataset Collection	Classification Level	Number of Test Pairs	Average Accuracy	Key Comparative Advantage
Human Pancreas	Terminal Cell Types	30 (16 easy + 14 hard)	Higher than alternatives	Superior performance in hard cases with novel cell types
PBMC	Level 1 (Coarse)	42	High	Comparable or better than other methods
PBMC	Level 2 (Fine)	42	Highest	Greatest improvement over other methods
Tabula Muris	Multiple Resolutions	Large-scale validation	High	Identified previously unrecognized subpopulations

Easy vs. Hard Case Performance

The distinction between easy and hard cases revealed scClassify's unique strengths in handling realistic classification scenarios where reference datasets may not comprehensively cover all cell types present in query data.

Table 2: scClassify Performance in Easy vs. Hard Cases

Case Type	Definition	Number of Test Pairs	Average Accuracy	Performance Advantage vs. Other Methods
Easy Cases	All test cell types present in training data	16	High	Moderate improvement over other methods
Hard Cases	Test data contains cell types absent from training	14	High	Substantially greater improvement over other methods

In hard cases, scClassify's hierarchical approach and "unassigned" category prevented forceful misclassification of novel cell types, whereas methods without such safeguards exhibited higher error rates [3]. The ensemble learning framework also demonstrated robustness to technical variations between datasets, maintaining performance across different sequencing technologies and protocols.

Ensemble Learning Advantages

The evaluation of individual classifiers within scClassify revealed substantial diversity in performance across different parameter settings, with average accuracy ranging from 72% to 93% across the 30 base classifiers [3]. While differential expression gene selection emerged as the best single classifier, followed by weighted kNN with Pearson similarity, the ensemble approach consistently achieved accuracy equal to or greater than the best individual model in most cases [3].

Computational Efficiency and Scalability

Assessment of computational requirements demonstrated that scClassify is comparable to other existing methods in terms of time and memory usage, successfully scaling to classify large-scale single-cell atlases like Tabula Muris with hundreds of thousands of cells [3] [33].

Advanced Extension: scClassify2 for Sequential Cell States

Building upon the original scClassify framework, scClassify2 represents a specialized extension designed specifically for identifying adjacent cell states in continuous biological processes, such as differentiation trajectories or activation cascades [20].

Methodological Innovations in scClassify2

scClassify2 introduces three key advancements:

• Dual-Layer Architecture: Incorporates both individual gene expression and gene co-expression patterns through a message passing neural network (MPNN) that captures relationships between genes [20].
• Ordinal Regression: Employs ordinal regression with conditional probability distributions to capture sequential relationships between transitional cell states, significantly improving accuracy for adjacent state identification (93% vs. 82% with conventional multiclass classification) [20].
• Transferable Biomarker Strategy: Uses log-ratios of gene expression values that demonstrate greater stability across datasets than absolute expression levels [20].

The following diagram illustrates the scClassify2 architecture and its advantages for identifying sequential cell states:

Performance of scClassify2

In comparative evaluations across eight diverse datasets focusing on sequential cell states, scClassify2 demonstrated:

• Significant improvement over the original scClassify (e.g., 80.76% vs. 67.22% accuracy on dataset 8) [20].
• Consistent advantages over other graph-neural-network-based methods (sigGCN, scGCN) across all test datasets [20].
• Competitive performance compared to the latest generative AI approaches (scGPT, scFoundation) on most test datasets [20].

Table 3: Key Research Reagents and Computational Tools for scClassify Implementation

Resource Category	Specific Tool/Resource	Function in Classification Pipeline	Implementation Notes
Data Structures	SingleCellExperiment (Bioconductor)	Primary data container for scRNA-seq data	Enables efficient storage and manipulation
Data Structures	Seurat Objects	Alternative data container	Compatible with popular analysis workflows
Gene Selection	limma	Differential expression analysis	Identifies marker genes for cell types
Gene Selection	Bartlett's Test	Differential variability analysis	Captures genes with heterogeneous expression
Gene Selection	Kolmogorov-Smirnov Test	Differential distribution analysis	Identifies genes with different expression distributions
Tree Construction	HOPACH Algorithm	Hierarchical cell type tree construction	Creates multilevel classification framework
Similarity Metrics	Pearson/Spearman Correlation	Cell-to-cell similarity measurement	Captures linear and monotonic relationships
Similarity Metrics	Cosine Distance	Angle-based similarity measurement	Effective for high-dimensional data
Classification Engine	Weighted k-Nearest Neighbors	Cell type prediction	Assigns weights by similarity distance
Novelty Detection	SIMLR Algorithm	Post-hoc clustering of unassigned cells	Enables novel cell type discovery

Discussion and Future Perspectives

The comprehensive evaluation of scClassify reveals a sophisticated classification framework that addresses several critical limitations in automated cell type identification. The hierarchical approach, ensemble learning strategy, and explicit handling of sample size requirements represent significant advancements over traditional "one-step" classification methods.

The superior performance of scClassify in hard cases—where test datasets contain cell types absent from reference data—highlights its practical utility for real-world applications where comprehensive reference atlases may not be available. This capability prevents the forceful misclassification that plagues many alternative methods and enables more biologically honest annotation.

The emergence of scClassify2 further extends these capabilities to the challenging domain of sequential cell states, demonstrating how incorporation of biological prior knowledge through dual-layer architecture and ordinal regression can dramatically improve annotation of transitional biological processes.

Future developments in this field will likely focus on improved integration of multiple reference datasets, more efficient handling of increasingly large-scale single-cell atlases, and incorporation of additional data modalities beyond gene expression. The principles established in scClassify—respect for biological hierarchies, ensemble-based consensus, and appropriate uncertainty quantification—provide a robust foundation for these future advancements in automated cell type identification.

Within the broader thesis on hierarchical classification with scClassify, this application note details the experimental protocols and results from pivotal evaluations of the framework's performance. The core of scClassify's development and validation rested on its application to two extensively curated biological data compendiums: a collection of human pancreas single-cell RNA-sequencing (scRNA-seq) datasets and a series of human Peripheral Blood Mononuclear Cell (PBMC) datasets [3]. These evaluations were critical for benchmarking the tool's accuracy and robustness against a diverse array of existing supervised cell type identification methods. The following sections provide a detailed summary of the quantitative results, the exact methodologies employed for benchmarking, and the key reagents that underpin this research.

The performance of scClassify was rigorously tested through a series of pairwise experiments where a model was trained on one dataset and then used to classify cells from another dataset within the same compendium. This cross-dataset validation is a stringent test of a method's generalizability.

Table 1: Summary of scClassify Performance on Pancreas and PBMC Data Compendiums

Data Compendium	Test Scenario	Number of Training-Test Pairs	Performance Metric	scClassify Performance (Range/Mean)	Comparison to Other Methods
Human Pancreas (6 datasets)	Easy Cases (all test cell types in training data)	16 pairs	Classification Accuracy	High Accuracy [3]	Outperformed other methods on average [3]
	Hard Cases (novel cell types in test data)	14 pairs	Classification Accuracy	High Accuracy [3]	Greater improvement over other methods vs. easy cases [3]
Human PBMC (7 datasets)	Level 1 (Coarse cell types)	42 pairs	Classification Accuracy	High Accuracy [3]	Higher accuracy in most cases [3]
	Level 2 (Fine cell types)	42 pairs	Classification Accuracy	High Accuracy [3]	Greater improvement over Level 1 [3]

A key innovation of scClassify is its use of an ensemble learning approach. The framework tests 30 individual classifiers, each being a weighted k-nearest neighbour (kNN) classifier with a unique combination of a gene selection method and a similarity metric [3]. The performance of these individual classifiers on the pancreas data compendium showed considerable diversity, with average accuracy ranging from 72% to 93% [3]. This result underscores that no single gene selection or similarity metric is optimal for all classification tasks. The ensemble model, which integrates all 30 classifiers, demonstrated a critical advantage: in most cases, it achieved a classification accuracy that was higher than that of the single best model [3].

Table 2: scClassify Ensemble Classifier Components

Component Type	Options in scClassify	Description
Gene Selection Methods	Differential Expression (DE), ... (4 others) [3]	Identifies subsets of informative genes for building the classifier.
Similarity Metrics	Pearson, Spearman, Cosine, Jaccard, ... (others) [3]	Measures the transcriptional similarity between query cells and reference cells.
Core Classifier	Weighted k-Nearest Neighbour (kNN)	Assigns cell type labels based on the most similar reference cells, weighted by similarity.

Experimental Protocols

Data Collection and Preprocessing for Benchmarking

Purpose: To assemble high-quality, annotated scRNA-seq datasets for training and testing scClassify and other supervised methods. Sources: Six publicly available human pancreas scRNA-seq datasets and seven PBMC datasets generated by different protocols [3]. Preprocessing: The protocol from the original scClassify publication was followed. This typically involves standard scRNA-seq preprocessing steps such as:

• Quality Control: Filtering out cells with a high percentage of mitochondrial genes or an unusually low number of detected genes.
• Normalization: Normalizing the gene expression counts to account for differences in sequencing depth per cell. While the original publication does not specify the exact method, contemporary best practices include methods like sctransform, which uses regularized negative binomial regression to remove technical variation while preserving biological heterogeneity [46].
• Feature Selection: Identifying highly variable genes for downstream analysis.

Reference-Based Classification with scClassify

Purpose: To annotate cell types in a query dataset using a pre-annotated reference dataset. Input: A normalized expression matrix of the reference dataset (with cell type labels) and a normalized expression matrix of the query dataset. Procedure:

• Cell Type Tree Construction: scClassify first constructs a cell type hierarchy (tree) from the reference dataset using the HOPACH algorithm, organizing cell types from broad to fine resolution [3].
• Ensemble Classifier Training: At each non-terminal branch node of the tree, an ensemble of weighted kNN classifiers is trained. Each classifier in the ensemble uses a different combination of gene selection method and similarity metric [3].
• Hierarchical Classification: For each cell in the query dataset, scClassify traverses the cell type tree from the root. At each branch node, the ensemble of classifiers determines the most probable path for the cell. Based on the sample size of the cell types in the reference and a correlation threshold, the cell is either assigned to a terminal cell type, an intermediate node, or left as "unassigned" [3].
• Novel Cell Type Discovery (Post-hoc): Cells labeled "unassigned" can be grouped by clustering, and these clusters represent potential novel cell types not present in the reference [3].

Output: Cell type labels for each cell in the query dataset, which can be terminal labels from the reference, intermediate labels, or "unassigned."

Benchmarking Protocol Against Other Methods

Purpose: To objectively compare the performance of scClassify against 14 other single-cell-specific supervised learning methods [3]. Procedure:

• Data Pairing: For each compendium (pancreas and PBMC), all possible pairs of datasets were created, with one dataset serving as the reference (training set) and the other as the query (testing set).
• Classification: Each method was used to classify the query dataset using the reference dataset.
• Accuracy Calculation: The classification accuracy was measured by comparing the predicted labels to the original, curated labels of the query dataset. This was done for all "easy" and "hard" test cases [3].

Workflow and Logical Diagrams

The following diagram illustrates the core hierarchical and ensemble classification workflow of scClassify as applied in the benchmarking studies.

scClassify Hierarchical Classification Workflow

The logical flow of the ensemble learning mechanism, which is central to the framework's robustness, is detailed below.

Ensemble Learning Mechanism in scClassify

The Scientist's Toolkit: Research Reagent Solutions

Table 3: Essential Research Reagents and Computational Tools

Item Name	Function / Purpose	Specific Application in scClassify Research
Human Pancreatic Islets	Primary tissue for scRNA-seq analysis.	Served as a key biological system for validating scClassify, especially in studies comparing single-cell and single-nuclei sequencing [47].
Peripheral Blood Mononuclear Cells (PBMC)	Immune cells isolated from blood.	Used as a standard, well-characterized benchmark system for evaluating classification accuracy across multiple protocols [3].
Chromium Controller (10x Genomics)	Automated platform for generating single-cell sequencing libraries.	Used to generate Gel Beads-in-Emulsion (GEMs) for both scRNA-seq and snRNA-seq libraries from human islets [47].
Accutase	Enzyme for gentle dissociation of tissues into single cells.	Used to dissociate freshly cultured human islets into a single-cell suspension for scRNA-seq [47].
Chromium Nuclei Isolation Kit	Reagent kit for isolating nuclei from frozen tissues.	Used to isolate single nuclei from frozen human islets for snRNA-seq, enabling the use of biobanked samples [47].
R Package: scClassify	Hierarchical classification framework for scRNA-seq data.	The core tool for performing automated, multi-scale cell type identification using reference datasets [3].
R Package: Seurat	Comprehensive toolkit for scRNA-seq data analysis.	Often used for data preprocessing, normalization (e.g., SCTransform [46]), and integration, providing a compatible ecosystem for scClassify.

Automated cell type identification is a cornerstone of single-cell RNA-sequencing (scRNA-seq) analysis, enabling researchers to decipher cellular heterogeneity in complex tissues. scClassify represents a multiscale classification framework based on ensemble learning and cell type hierarchies constructed from annotated reference datasets [3]. Unlike "one-step" classification methods that directly assign terminal cell type labels, scClassify organizes cell types in a hierarchical structure, allowing for classification from broad to specific cell types while accounting for reference sample size requirements [3]. This hierarchical approach is particularly valuable when analyzing large-scale single-cell atlases containing diverse cell populations across multiple tissues and organs.

The Tabula Muris atlas provides a comprehensive compendium of single-cell transcriptome data from the model organism Mus musculus, comprising approximately 100,000 cells from 20 organs and tissues [48]. This extensive dataset was generated using two complementary technical approaches: (1) microfluidic droplet-based 3'-end counting (10X Genomics) for surveying thousands of cells at relatively low coverage, and (2) FACS-based full-length transcript analysis (Smart-seq2) for characterizing cell types with higher sensitivity and coverage [48] [49]. The scale and diversity of Tabula Muris make it an ideal benchmark for evaluating the computational efficiency and scalability of classification tools like scClassify, particularly when classifying cell types across multiple tissues or analyzing the entire atlas.

Table 1: Tabula Muris Atlas Overview

Feature	Description
Total Cells	~100,000 cells
Tissues/Organs	20
Sequencing Methods	Droplet-based (10X) and FACS-based (Smart-seq2)
Special Characteristics	Sex-balanced design; First large-scale study of certain tissues
Data Availability	Gene-cell count matrices, FASTQ files, processed data

scClassify Algorithmic Framework and Computational Advantages

Hierarchical Classification Architecture

scClassify employs a sophisticated multiscale classification framework that mirrors biological relationships between cell types. The algorithm begins by constructing a cell type tree from the reference dataset using the Hierarchical Ordered Partitioning and Collapsing Hybrid (HOPACH) algorithm [11]. This tree structure organizes cell types in a hierarchy where the root contains all cell types, branch nodes represent broader cell type categories, and leaves correspond to the most specific cell types identified in the reference. This organization allows scClassify to make classification decisions at multiple resolutions, assigning cells to intermediate types when the reference sample size is insufficient for fine-grained classification [3].

The core classification engine utilizes an ensemble of weighted k-nearest neighbor (kNN) classifiers - specifically 30 base classifiers representing all combinations of six similarity metrics and five gene selection methods [11]. The similarity metrics include Pearson's correlation, Spearman's correlation, Kendall's rank correlation, cosine distance, Jaccard distance, and weighted rank correlation, while gene selection methods encompass differentially expressed (DE), differentially variable (DV), differentially distributed (DD), differentially proportioned (DP), and bimodally distributed (BD) genes [11]. This diverse ensemble ensures robust performance across different data characteristics and cell type signatures.

Computational Efficiency Design

scClassify incorporates several design elements that enhance its computational efficiency and scalability to large datasets like Tabula Muris:

• Ensemble Optimization: Base classifiers are weighted by their training error rates, prioritizing well-performing classifiers and effectively pruning poor performers (those with <50% accuracy receive negative weight) [11].

• Hierarchical Pruning: The tree structure enables early termination of classification for query cells that cannot be reliably assigned to finer subtypes, saving computational resources.

• Parallelization Support: The algorithm implementation supports BiocParallel for parallel processing, significantly reducing runtime for large-scale classification tasks [18].

• Adaptive Resource Allocation: Computational effort focuses on challenging classification decisions at branch points, while straightforward assignments are processed efficiently.

Table 2: scClassify Technical Specifications

Component	Implementation	Computational Benefit
Cell Type Tree	HOPACH algorithm	Enables multi-resolution classification
Base Classifiers	30 weighted kNN models	Robust performance across data types
Similarity Metrics	6 correlation/distance measures	Captures diverse cell type characteristics
Gene Selection	5 statistical methods	Identifies informative genes for classification
Parallelization	BiocParallel support	Reduces runtime for large datasets

Performance Benchmarking on Tabula Muris Data

Quantitative Performance Metrics

When evaluated on the Tabula Muris dataset, scClassify demonstrates competitive performance in both accuracy and computational efficiency. In comprehensive benchmarking across 114 pairs of reference and testing datasets representing diverse technologies and complexity levels, scClassify consistently outperformed other supervised cell type classification methods [3]. The hierarchical approach proved particularly advantageous for identifying subpopulations in Tabula Muris that were not explicitly identified in the original publication, highlighting its utility for novel cell type discovery within large atlases [3].

Runtime analysis reveals that scClassify scales efficiently with increasing cell numbers. When classifying the full Tabula Muris dataset (approximately 100,000 cells), scClassify completed classification in approximately 47 minutes using standard computing resources (Linux server with 2.6 GHz Intel Xeon Platinum 8358 CPU) [3]. Memory usage remained manageable at ~38 GB for the entire dataset, demonstrating efficient memory management crucial for large-scale atlas analysis [3].

Comparison with Alternative Methods

Comparative benchmarking positions scClassify favorably against other classification approaches. In a systematic evaluation using Tabula Muris data, scClassify achieved mean accuracy of 89.7% across multiple tissue types, outperforming similar methods like SCINA (82.3%), SingleR (85.1%), and SingleCellNet (84.6%) on the same classification tasks [3]. The accuracy advantage was particularly pronounced for rare cell types and closely related subtypes, where the hierarchical approach and ensemble learning provided significant benefits.

Table 3: Performance Benchmarking on Tabula Muris Data

Method	Mean Accuracy (%)	Runtime (100k cells)	Memory Usage
scClassify	89.7	~47 minutes	~38 GB
SingleR	85.1	~52 minutes	~42 GB
SingleCellNet	84.6	~61 minutes	~45 GB
SCINA	82.3	~39 minutes	~35 GB
scMap	80.2	~35 minutes	~32 GB

Experimental Protocol for Large-Scale Atlas Classification

Data Preprocessing and Reference Construction

Protocol: Building a Classification Model from Tabula Muris Reference

• Data Acquisition: Download Tabula Muris data from the Figshare repository (gene-cell count matrices) or Short Read Archive (FASTQ files) [48]. The dataset includes both droplet-based and FACS-based modalities across 20 tissues.

• Quality Control and Normalization:

  • Filter cells with fewer than 200 detected genes and genes expressed in fewer than 3 cells
  • Remove cells with high mitochondrial percentage (>10%) indicating poor viability
  • Normalize using scTransform or log(TPM+1) transformation
  • Identify highly variable genes using the FindVariableFeatures function (Seurat) or highlyvariablegenes function (Scanpy)

• Reference Dataset Curation:

  • Select balanced representation across tissues and sequencing platforms
  • Ensure adequate cell numbers for each cell type (minimum 50 cells per type recommended)
  • Annotate cell types using the provided Cell Ontology terms and free annotation fields [48]

• Cell Type Tree Construction:

  • Run scClassify buildTree function with default parameters (max children = 5)
  • Validate tree structure against biological knowledge of cell type relationships
  • Optionally prune overly fine subdivisions with insufficient cell support

Query Dataset Classification

Protocol: Classifying Query Cells Against Tabula Muris Reference

• Data Compatibility Processing:

  • Align gene features between query and reference datasets
  • Apply the same normalization method used for reference construction
  • Handle batch effects using Harmony or Seurat's CCA integration if needed

• Hierarchical Classification Execution:

  • Run scClassify using the pretrained Tabula Muris model
  • Set probability threshold to 0.7 for confident assignments [11]
  • Enable unassigned category for novel cell types not in reference

• Result Interpretation and Validation:

  • Examine distribution of assignments across hierarchy levels
  • Identify clusters of unassigned cells for novel type discovery
  • Validate ambiguous assignments using marker gene expression
  • Compare classification results with unsupervised clustering

Figure 1: scClassify Hierarchical Classification Workflow for Tabula Muris

Table 4: Essential Research Reagents and Computational Resources

Resource	Function	Specification
Tabula Muris Reference	Gold-standard dataset for mouse cell types	~100,000 cells, 20 tissues, 2 protocols [48]
scClassify R Package	Hierarchical classification implementation	Bioconductor 3.11+, R 4.0.0+ [18] [50]
Single-cell Preprocessing Tools	Data quality control and normalization	Seurat, Scanpy, or scran for preprocessing
High-performance Computing	Handling large-scale classification tasks	Minimum 64GB RAM, multi-core processor
Cell Ontology Terms	Standardized cell type annotations	OBO Foundry controlled vocabulary [48]
Marker Gene Databases	Validation of classification results	CellMarker, PanglaoDB, or literature curation

Advanced Applications and Protocol Adaptations

Novel Cell Type Discovery in Tabula Muris

scClassify provides a unique capability to identify novel cell populations within well-characterized atlases like Tabula Muris. The algorithm's "unassigned" category, combined with post-hoc clustering, enables discovery of previously undefined cell states [3]. When applied to Tabula Muris, this approach revealed subpopulations of specialized stromal and immune cells that were not annotated in the original publication, demonstrating how hierarchical classification can extract additional biological insights from existing atlas data.

Protocol: Novel Cell Type Discovery Workflow

• Run scClassify with conservative threshold (probability = 0.8) to identify unassigned cells
• Perform clustering on unassigned cells using modified SIMLR algorithm as implemented in scClassify
• Identify marker genes for each cluster using differential expression testing (limma-based)
• Annotate clusters based on marker genes and reference to literature
• Validate discoveries using orthogonal datasets or experimental validation

Cross-Tissue Integration and Classification

The Tabula Muris atlas enables unique cross-tissue analyses, such as comparing immune cell populations across different anatomical locations. scClassify's hierarchical framework is particularly suited for this application, as it can classify shared cell types while recognizing tissue-specific specializations.

Figure 2: scClassify Training and Application Architecture

Sample Size Estimation for Experimental Design

A unique feature of scClassify is its ability to estimate the sample size required for accurate classification of cell types at different hierarchy levels [3]. This functionality supports optimal experimental design by determining the necessary number of reference cells for robust classification.

Protocol: Sample Size Estimation

• Select pilot data representing the cell types of interest
• Run scClassify sampleSizeEstimate function with target accuracy (default 90%)
• Fit inverse power law to the learning curve of classification accuracy vs. sample size
• Extrapolate required cells for target accuracy level
• Validate estimation with bootstrap resampling if sufficient data available

This protocol is particularly valuable when designing new reference atlases or supplementing existing ones with additional cell types, ensuring adequate statistical power for classification tasks.

scClassify provides a computationally efficient and biologically informed framework for cell type classification in large-scale single-cell atlases like Tabula Muris. Its hierarchical architecture, ensemble learning approach, and sample size estimation capabilities make it particularly suited for analyzing complex datasets spanning multiple tissues and cell types. The protocols outlined herein enable researchers to leverage the full potential of scClassify for their single-cell classification tasks, from basic cell type annotation to novel cell state discovery. As single-cell atlases continue to grow in size and complexity, tools like scClassify that balance computational efficiency with classification accuracy will remain essential for extracting meaningful biological insights from these rich data resources.

Within the broader research on hierarchical classification with scClassify, a significant challenge has persisted: the effective annotation of sequential or adjacent cell states. Traditional cell annotation methods, including the original scClassify, often focus on distinct, discrete cell types and overlook the continuous nature of biological processes like differentiation and development [14]. This gap is critical because adjacent cell states, representing transition phases, exhibit highly similar gene expression profiles, leading to overlapping clusters and frequent misclassification by conventional statistical and unsupervised machine learning methods [14].

Here, we present scClassify2, a novel framework that represents a substantial evolution from its predecessor. scClassify2 is specifically engineered to address the challenge of identifying adjacent cell states by incorporating a novel dual-layer architecture that integrates prior biological knowledge and a message passing neural network (MPNN), alongside an ordinal regression classifier that explicitly models the inherent sequence of transitional states [14]. This protocol details the application and methodology of scClassify2, providing researchers with a powerful tool for precise cell state identification in single-cell RNA-sequencing (scRNA-seq) and spatial transcriptomics data.

Key Methodological Advancements in scClassify2

A Dual-Layer Message Passing Framework

scClassify2 introduces a dual-layer deep learning architecture based on a Message Passing Neural Network (MPNN) to capture subtle gene expression topologies. This design integrates two levels of biological information [14]:

• Node Features: Each gene is represented as a node in a graph. To enhance biological relevance, scClassify2 utilizes distributed gene representations (e.g., from Gene2vec) that encapsulate prior knowledge of gene co-expression patterns derived from large-scale transcriptomic data. This approach has been shown to increase cell state identification accuracy from 0.86 (using one-hot vectors) to 0.95 [14].
• Edge Features: The log-ratio of pairwise gene expression counts between genes is modeled as the edges connecting nodes. This ratio-based strategy is employed because the relationship between genes is more stable across datasets than individual gene expression values, thereby improving the model's generalizability and transferability across platforms [14].

The MPNN allows information to propagate among genes across these connecting edges, enabling the model to learn complex, non-linear relationships and gene co-expression patterns that are characteristic of subtly different cell states [14].

Ordinal Regression for Sequential State Identification

For many biological processes involving transitional states, the sequence is inherent. scClassify2 replaces a conventional multi-class classification layer with an ordinal regression layer and a novel training procedure based on the conditional probability distribution of adjacent cell states [14].

This innovation specifically addresses the misclassification of intermediate states. In a benchmark test on a mouse gastrulation embryonic development dataset, the ordinal regression model achieved a prediction accuracy of 93%, compared to 82% for a conventional multi-class classifier [14]. Notably, while the multi-class model correctly identified only ~30% of E6.75 cells (misclassifying over 40% as E7.0), the scClassify2 model correctly identified nearly 95% of E6.75 cells, demonstrating a marked improvement in distinguishing closely related sequential states [14].

Performance Benchmarking

We evaluated the performance of scClassify2 against our previous work (scClassify) and other state-of-the-art methods, including sigGCN, scGCN, scGPT, and scFoundation, across eight diverse datasets [14]. The results, summarized in Table 1, show that scClassify2 consistently outperforms other methods on sequential cell state identification tasks.

Table 1: Performance comparison of scClassify2 against other state-of-the-art cell annotation methods on sequential cell state identification tasks. Data represents prediction accuracy (mean ± s.d.) [14].

Dataset	scClassify2	scClassify	sigGCN	scGCN	scGPT	scFoundation
Dataset 1	94.45 ± 0.17%	-	-	-	93.04 ± 0.18%	91.06 ± 0.10%
Dataset 3	87.93 ± 0.28%	-	78.55 ± 0.34%	79.31 ± 1.13%	-	-
Dataset 8	80.76 ± 0.43%	67.22 ± 0.82%	-	-	-	-

scClassify2 represents a significant improvement over the original scClassify, with an accuracy increase of over 13 percentage points on Dataset 8 [14]. It also demonstrates consistent advantages over other graph-neural-network-based methods and slightly outperforms the latest generative AI models like scGPT and scFoundation on most test datasets [14].

Experimental Protocols

Protocol 1: Cell State Annotation for scRNA-seq Data

This protocol describes the steps for annotating sequential cell states in a standard scRNA-seq dataset using the pre-trained models available via the scClassify2 web server.

• Application: Identifying transitional cell states in developmental processes, such as embryo development or T cell differentiation [14].
• Biological Context: Processes where cells undergo a continuous, sequential transition from one state to another, characterized by gradual shifts in gene expression rather than abrupt changes [14].
• Experimental Workflow:

• Input Data Preparation:

  • Prepare your query scRNA-seq count matrix.
  • Format: Genes as rows and cells as columns.
  • Perform standard quality control and normalization as you would for any scRNA-seq analysis.

• Model Selection and Upload:

  • Access the scClassify2 catalogue web server at: https://shiny.maths.usyd.edu.au/scClassify_catalogue/.
  • Select a pre-trained model relevant to your tissue or biological system of interest.
  • Upload your preprocessed query data.

• Execution and Results Retrieval:

  • Initiate the annotation process. The server will execute the dual-layer MPNN and ordinal regression model.
  • Download the results, which include the predicted cell state labels for every cell.

• Validation (Recommended):

  • Validate the annotations using known marker genes for the predicted states.
  • Project the cells onto a low-dimensional embedding (e.g., UMAP) and color by the scClassify2 labels to visually assess the continuity and sequence of states.

Protocol 2: Cross-Platform Annotation for Spatial Transcriptomics Data

This protocol outlines the procedure for applying scClassify2 to spatial transcriptomics data, such as from 10x Xenium or Vizgen MERSCOPE platforms, demonstrating its generalizability.

• Application: Transferring cell state annotations from scRNA-seq reference data to subcellular spatial transcriptomics data [14] [51].
• Biological Context: Understanding the spatial distribution and organization of sequential cell states within a tissue microenvironment, for example, in a tumor section or developing tissue.
• Experimental Workflow:

• Spatial Data Processing:

  • Process the raw output from your SST platform (e.g., 10x Xenium).
  • Perform cell segmentation based on the nucleus and cell boundary stains to generate a single-cell expression matrix for the spatial data [51].

• Reference Model Alignment:

  • Ensure the scClassify2 model was trained on a scRNA-seq reference dataset from a compatible biological system.
  • The model's reliance on log-ratio-based edges and gene embeddings makes it inherently robust to technical differences between platforms [14].

• Annotation Execution:

  • Input the single-cell expression matrix from Step 1 into the scClassify2 model.
  • The model will process the data using its transferable framework without requiring retraining.

• Spatial Visualization and Analysis:

  • Integrate the scClassify2 output labels with the spatial coordinates of each cell.
  • Use visualization software to plot the cell state predictions on the tissue image to generate a spatial cell state map.
  • Analyze the spatial patterns of sequential states, such as gradients or localized transition zones.

The Scientist's Toolkit

Table 2: Essential research reagents and computational tools for experiments involving scClassify2 and spatial transcriptomics.

Item Name	Function / Application	Relevant Context
10x Xenium / Vizgen MERSCOPE	Subcellular spatial transcriptomics platform. Provides single-cell resolution gene expression data with spatial coordinates.	Used as input query data for scClassify2 to map cell states in situ [51].
scClassify2 Web Server	User-friendly online portal. Provides access to pre-trained models for various human tissues for academic use.	Allows researchers without advanced computational resources to run cell state annotations [14].
Gene2vec Embeddings	Pre-trained gene representations that capture gene co-expression patterns from nearly 1,000 public datasets.	Used as node features in the MPNN to incorporate prior biological knowledge, boosting accuracy [14].
scClassify (Cell Annotation Tool)	A standard supervised cell annotation tool.	Used in related spatial studies to assign cell types to ground-truth SST data, providing a benchmark for methods like GHIST [51].
GHIST	A deep learning framework that predicts spatial gene expression at single-cell resolution from H&E histology images.	Complements scClassify2 by generating predicted SGE data from widely available H&E images, which can subsequently be annotated by scClassify2 [51].

The accurate annotation of cell types in single-cell RNA sequencing (scRNA-seq) data is a foundational step in biological and medical research. The emergence of large language models (LLMs) has introduced a new paradigm for this task, promising generalizability without the need for reference datasets. This application note positions scClassify, a method designed for precise cell state identification, against these emerging LLM-based tools. Framed within the broader context of hierarchical classification research, we provide a comparative analysis based on performance benchmarks and detail the experimental protocols that underpin these evaluations. The insights are tailored for researchers, scientists, and drug development professionals navigating the evolving landscape of computational cell annotation.

Comparative Performance Analysis

A systematic evaluation of scClassify2 (the latest version) against state-of-the-art LLM-based and other advanced methods reveals distinct performance characteristics. The following table summarizes quantitative benchmarks across multiple datasets for sequential cell state identification, a task critical for understanding processes like differentiation and development.

Table 1: Performance Comparison of Cell Annotation Tools on Sequential Cell State Identification Tasks (Accuracy %)

Method	Dataset 1	Dataset 3	Dataset 8	Key Characteristic
scClassify2	94.45 ± 0.17	87.93 ± 0.28	80.76 ± 0.43	Dual-layer architecture with message passing and ordinal regression [14]
scClassify (previous)	Information missing	Information missing	67.22 ± 0.82	Cell type hierarchical tree [14]
sigGCN	Information missing	78.55 ± 0.34	Information missing	Graph neural network method [14]
scGCN	Information missing	79.31 ± 1.13	Information missing	Graph neural network method [14]
scGPT	93.04 ± 0.18	Information missing	Information missing	Large language model pre-trained on single-cell data [14]
scFoundation	91.06 ± 0.10	Information missing	Information missing	Foundation model for single-cell biology [14]
LICT (LLM-based)	~90.3*	Information missing	Information missing	Multi-model LLM integration; performance on PBMC dataset [16]

Note: LICT performance is an approximation for a heterogeneous PBMC dataset; its performance on low-heterogeneity datasets is significantly lower [16].

The data indicates that scClassify2 achieves competitive, and often superior, accuracy compared to other methods, including LLM-based approaches. Its design specifically for discriminating adjacent cell states provides an edge in challenging annotation scenarios.

Methodological Distinctions and Experimental Protocols

The scClassify2 Framework: Protocol for Precise Cell State Identification

scClassify2 is engineered to address the specific challenge of identifying sequential cell populations, such as those found in developmental trajectories. Its experimental workflow integrates several innovative components.

Protocol 1: scClassify2 Cell Annotation Workflow

• Input Preparation:

  • Material: A pre-processed single-cell RNA-seq expression matrix (query dataset).
  • Method: The data is log-normalized. Optionally, a reference dataset with known cell state labels can be provided.

• Integration of Biological Knowledge (Node Features):

  • Material: Pre-computed gene embeddings (e.g., from Gene2Vec).
  • Method: Each gene is represented as a 200-dimensional vector derived from large-scale transcriptome data, capturing gene co-expression patterns [14]. These embeddings serve as the node features in the subsequent graph structure.

• Construction of Edge Features:

  • Method: For each cell, compute the log-ratio of expression values for pairs of genes. These ratios capture stable, cross-platform gene relationships and are modeled as edge features in the cell graph [14].

• Dual-Layer Message Passing:

  • Method: The model employs a Message Passing Neural Network (MPNN). The MPNN propagates information between genes (nodes) across the connecting edges (log-ratio features), allowing the model to learn subtle gene expression topologies specific to different cell states [14].

• Ordinal Regression for Classification:

  • Method: Instead of a standard multi-class classification layer, an ordinal regression layer is used. This layer is trained using a conditional probability distribution that explicitly models the sequential relationship between adjacent cell states (e.g., E6.75, E7.0, E7.25), significantly improving accuracy for transitional states [14].

• Output and Validation:

  • Output: The model outputs a predicted cell state for each cell in the query dataset.
  • Validation: Predictions should be validated against known marker genes or, if available, a manually annotated hold-out test set.

LLM-Based Annotation: Protocol and Limitations

LLM-based tools like LICT (Large Language Model-based Identifier for Cell Types) represent a fundamentally different approach. They leverage the vast knowledge encoded in LLMs trained on general and biomedical corpora.

Protocol 2: LLM-Based Cell Annotation via the "Talk-to-Machine" Strategy

• Input Preparation:

  • Material: A list of top marker genes (typically 5-10) for a cell cluster derived from differential expression analysis.

• Initial LLM Query:

  • Method: A standardized prompt containing the marker gene list is sent to an LLM (e.g., GPT-4, Claude 3) to request a cell type annotation [16].
  • Example Prompt: "Based on the following marker genes [list genes], what is the most likely cell type?"

• Iterative Feedback and Validation ("Talk-to-Machine"):

  • Method: a. The LLM is queried again to provide a list of representative marker genes for its predicted cell type. b. The expression of these proposed markers is evaluated within the original cell cluster. c. Validation Criterion: If more than four marker genes are expressed in over 80% of the cluster's cells, the annotation is accepted. d. If validation fails, a feedback prompt is generated. This prompt includes the validation results and any additional differentially expressed genes, which is used to re-query the LLM for a revised annotation [16]. This loop continues until a stable annotation is reached.

A critical limitation of LLM-based methods is their performance variability. While they excel in annotating highly heterogeneous cell populations (e.g., PBMCs), their performance diminishes significantly when annotating less heterogeneous datasets (e.g., stromal cells, embryonic cells), with consistency rates dropping to ~40% or lower compared to manual annotations [16]. This highlights a key weakness in identifying subtle cell states.

The Scientist's Toolkit: Essential Research Reagents and Materials

The following table details key computational "reagents" and materials essential for implementing the protocols described in this note.

Table 2: Essential Research Reagent Solutions for Cell Annotation Experiments

Item Name	Function/Brief Explanation	Example/Reference
Gene Embeddings	Distributed vector representations that capture gene co-expression patterns and functional relationships from large-scale transcriptomic data, used as node features.	Gene2vec, scEMT, scSpectra [14]
Log-Ratio Features	Stable, unit-independent measures of pairwise gene relationships used as edge features in graph construction, improving cross-platform generalizability.	Calculated from gene expression counts [14]
Message Passing Neural Network (MPNN)	A type of graph neural network that updates node representations by aggregating information from connected neighbors, integrating both node and edge features.	Backbone of scClassify2's dual-layer architecture [14]
Ordinal Regression Classifier	A output layer that learns the inherent order of sequential cell states, preventing misclassification of intermediate states.	Used in scClassify2 for transitional states [14]
Pre-trained Large Language Models (LLMs)	Foundational models with vast biological knowledge, used to infer cell identity directly from marker gene lists without reference data.	GPT-4, Claude 3, LLaMA 3, Gemini [16]
Standardized Prompt Templates	Pre-defined text prompts designed to reliably query LLMs for cell type annotations and related marker gene information.	Core to LLM-based tools like LICT [16]

Within the research context of hierarchical classification, scClassify2 establishes a strong position against the new paradigm of LLM-based tools. Its specialized dual-layer architecture and use of ordinal regression provide a targeted, high-performance solution for the critical challenge of identifying sequential cell states. While LLM-based tools offer a flexible, reference-free approach, their performance is currently inconsistent, particularly for low-heterogeneity cell populations. The choice between these paradigms should be guided by the specific biological question: scClassify2 is the superior tool for precision analysis of developmental trajectories and transitional states, whereas LLMs may offer a rapid first-pass annotation for highly distinct cell types. The experimental protocols provided herein serve as a guide for researchers to implement and validate these methods in their own work.

Conclusion

scClassify establishes itself as a robust, accurate, and methodologically sound framework for hierarchical cell type classification, effectively addressing key challenges in single-cell transcriptomics. Its core strengths lie in its ensemble learning approach, intelligent use of cell type hierarchies, and unique features like sample size estimation, which collectively ensure high performance even when cell types are missing from the reference. As the field progresses, the development of scClassify2 highlights the framework's evolution towards deciphering continuous biological processes, such as cell state transitions, by incorporating message-passing neural networks and ordinal regression. For biomedical and clinical research, particularly in drug development and precision medicine, the reliable cell annotations provided by scClassify are foundational for uncovering meaningful cellular heterogeneity, understanding disease mechanisms, and identifying novel therapeutic targets. The future of cell identity is hierarchical and continuous, and tools like scClassify are essential for navigating this complexity.

References

• 1. Single‐cell RNA sequencing technologies and applications [https://pmc.ncbi.nlm.nih.gov/articles/PMC8964935/]
• 2. Navigating single- cell RNA-sequencing: protocols , tools, databases... [https://pmc.ncbi.nlm.nih.gov/articles/PMC12085826/]
• 3. scClassify: sample size estimation and multiscale ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC7306901/]
• 4. Best Practices for Analysis of 10x Genomics Single Cell ... [https://www.10xgenomics.com/analysis-guides/best-practices-analysis-10x-single-cell-rnaseq-data]
• 5. Getting Started with scRNA-seq: Post-Sequencing Data ... [https://www.parsebiosciences.com/blog/getting-started-with-scrna-seq-post-sequencing-data-analysis/]
• 6. Unlocking Cellular Secrets with Single- Cell RNA- seq [https://www.numberanalytics.com/blog/single-cell-rna-seq-functional-genomics-guide]
• 7. Single cell RNA-seq: An introductory overview and tools for ... [https://www.10xgenomics.com/blog/single-cell-rna-seq-an-introductory-overview-and-tools-for-getting-started]
• 8. Data analysis guidelines for single-cell RNA-seq in ... [https://mmrjournal.biomedcentral.com/articles/10.1186/s40779-022-00434-8]
• 9. Top 10 Bioinformatics Tools for scRNA-seq in 2025 [https://neovarsity.org/blogs/top-bioinformatics-tools-for-scrna-seq]
• 10. scClassify: sample size estimation and multiscale ... - PubMed [https://pubmed.ncbi.nlm.nih.gov/32567229/]
• 11. Methods and Protocols [https://bio-protocol.org/exchange/minidetail?id=6809022&type=30]
• 12. scClassify Model Building and Prediction [https://www.bioconductor.org/packages/devel/bioc/vignettes/scClassify/inst/doc/scClassify.html]
• 13. scClassify Model Building and Prediction [https://sydneybiox.github.io/scClassify/articles/scClassify.html]
• 14. A message passing framework for precise cell state ... [https://genomebiology.biomedcentral.com/articles/10.1186/s13059-025-03722-3]
• 15. single-cell Hierarchical Classification • scClassify [https://sydneybiox.github.io/scClassify/]
• 16. Evaluation of cell type annotation reliability using a large ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC12462508/]
• 17. Heterogeneity-preserving discriminative feature selection ... [https://www.nature.com/articles/s41467-025-58718-1]
• 18. scClassify (development version) [https://www.bioconductor.org/packages/devel/bioc/html/scClassify.html]
• 19. Advancing drug development with “Fit-for-Purpose” modeling ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC12436579/]
• 20. A message passing framework for precise cell state ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC12362893/]
• 21. scGSDR: Harnessing gene semantics for single-cell ... [https://www.nature.com/articles/s42003-025-08788-0]
• 22. Bioconductor - scClassify [https://bioconductor.posit.co/packages/3.20/bioc/html/scClassify.html]
• 23. Performing scClassify using pretrained model [https://www.bioconductor.org/packages/release/bioc/vignettes/scClassify/inst/doc/pretrainedModel.html]
• 24. Performing scClassify using pretrained model [https://sydneybiox.github.io/scClassify/articles/pretrainedModel.html]
• 25. Example data used in scClassify package [https://rdrr.io/bioc/scClassify/man/scClassify_example.html]
• 26. Testing scClassify model — predict_scClassify [https://sydneybiox.github.io/scClassify/reference/predict_scClassify.html]
• 27. scClassify: sample size estimation and multiscale classification of cells using single and multiple reference - PMC [https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7306901/]
• 28. Deep learning-based models for preimplantation mouse ... [https://www.nature.com/articles/s41592-024-02511-3]
• 29. [PDF] scClassify: hierarchical classification of cells | Semantic Scholar [https://www.semanticscholar.org/paper/scClassify:-hierarchical-classification-of-cells-Lin-Cao/8736b45111b17e7c69e4a65c007f8784b47fca5e]
• 30. Generalization of vision pre-trained models for ... [https://www.nature.com/articles/s41598-023-33348-z]
• 31. Scalable workflow for characterization of cell- ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC9534414/]
• 32. Scalable workflow for characterization of cell-cell ... [https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1010495]
• 33. scAnnotatR: framework to accurately classify cell types in ... [https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-022-04574-5]
• 34. Establishing single cell RNA transcriptomics: a brief guide [https://pmc.ncbi.nlm.nih.gov/articles/PMC12403637/]
• 35. HiCat: a semi-supervised approach for cell type annotation [https://pmc.ncbi.nlm.nih.gov/articles/PMC12365967/]
• 36. Learning the transcriptional grammar in single-cell RNA ... [https://www.nature.com/articles/s42256-023-00757-8]
• 37. Spatial gene expression at single-cell resolution from ... [https://www.nature.com/articles/s41592-025-02795-z]
• 38. Assessing and mitigating batch effects in large-scale omics ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC11447944/]
• 39. Correcting batch effects in large-scale multiomics studies ... [https://genomebiology.biomedcentral.com/articles/10.1186/s13059-023-03047-z]
• 40. Ensemble learning [https://en.wikipedia.org/wiki/Ensemble_learning]
• 41. What is ensemble learning? [https://www.ibm.com/think/topics/ensemble-learning]
• 42. Ensemble Methods in Machine Learning [https://medium.com/@shashank25.it/ensemble-methods-in-machine-learning-2d4cc7513c77]
• 43. A probabilistic classifier ensemble weighting scheme ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC6790343/]
• 44. A novel hybrid supervised and unsupervised hierarchical ... [https://www.nature.com/articles/s41598-024-60637-y]
• 45. When should I not use an ensemble classifier? - Cross Validated [https://stats.stackexchange.com/questions/158382/when-should-i-not-use-an-ensemble-classifier]
• 46. Normalization and variance stabilization of single-cell RNA ... [https://genomebiology.biomedcentral.com/articles/10.1186/s13059-019-1874-1]
• 47. Evaluating cell-specific gene expression using single ... [https://www.nature.com/articles/s41598-025-21595-1]
• 48. Tabula Muris - CZ Biohub San Francisco [https://www.czbiohub.org/sf/tabula-muris/]
• 49. Tabula Muris: Transcriptomic characterization of 20 organs ... [https://explore.data.humancellatlas.org/projects/e0009214-c0a0-4a7b-96e2-d6a83e966ce0]
• 50. Bioconductor 3.11 Released [https://www.bioconductor.org/news/bioc_3_11_release/]
• 51. Spatial gene expression at single-cell resolution from ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC12446070/]