Automating Single-Cell Annotation: A Comprehensive Guide to SingleR for Precision Biology

Abstract

This article provides a complete resource for researchers and drug development professionals seeking to implement automated, reference-based cell type annotation using the SingleR package. It covers foundational concepts, step-by-step methodologies from data preparation to result interpretation, advanced optimization strategies for computational efficiency, and rigorous validation techniques. By comparing SingleR with emerging approaches like large language model-based tools, this guide empowers scientists to generate reliable, reproducible cell annotations, thereby accelerating discoveries in immunology, oncology, and clinical research.

What is SingleR and Why is it Revolutionizing Single-Cell RNA-Seq Analysis?

Single-cell RNA sequencing (scRNA-seq) has revolutionized biological research by enabling the analysis of gene expression patterns at the individual cell level, revealing unprecedented insights into cellular heterogeneity [1] [2]. Within this analytical pipeline, cell type annotation—the process of assigning identity labels to individual cells based on their gene expression profiles—stands as a crucial step for understanding cellular composition and function in complex biological systems [3]. Traditionally, this process has relied predominantly on manual annotation, where domain experts assign cell identities through visual inspection of cluster patterns and expression of known marker genes [4]. While this approach benefits from expert biological knowledge, it introduces significant challenges related to subjectivity and limited scalability that become increasingly problematic as dataset sizes grow into the hundreds of thousands of cells [5].

The inherent limitations of manual annotation have stimulated the development of automated methods, with reference-based approaches like SingleR emerging as powerful alternatives [5] [6]. These methods compare cells in a new dataset against curated reference profiles of known cell types, assigning each cell to the reference type that its expression profile most closely matches [5]. This automated paradigm offers the potential to overcome the key constraints of manual approaches while maintaining biological accuracy. This Application Note examines the specific limitations of manual cell annotation and provides detailed protocols for implementing reference-based annotation using SingleR, framed within the context of a broader research thesis on robust, scalable cell identification methods.

The Limitations of Manual Cell Annotation

Subjectivity and Expert Dependence

The manual annotation process is inherently subjective, with outcomes heavily dependent on the annotator's specific expertise and prior knowledge. This expert dependence introduces substantial variability in annotation results, even when highly experienced researchers analyze identical datasets [3]. Studies comparing manual annotations across different experts have revealed significant discrepancies, particularly for cell populations with ambiguous or overlapping marker expression patterns [3]. For instance, in analyses of stromal cells from mouse organs, manual annotations demonstrated poor reliability, with objective credibility evaluations finding that none of the manual annotations met established confidence thresholds [3]. This subjectivity problem is compounded by the context-specific nature of marker gene expression, where the same gene may serve as a marker for different cell types in different tissues or biological contexts.

Scalability Constraints in Large Datasets

The labor-intensive nature of manual annotation creates severe scalability constraints when dealing with the increasingly large datasets generated by modern single-cell technologies [1] [5]. As dataset sizes grow from thousands to millions of cells, the time and resources required for comprehensive manual annotation become prohibitive. This scalability limitation is not merely an inconvenience—it fundamentally constrains research progress by creating analytical bottlenecks that delay insights and discoveries. Furthermore, manual approaches struggle with cellular heterogeneity within seemingly uniform populations, often failing to distinguish closely related cell subtypes without targeted investigation [1]. The lack of standardization in manual annotation also creates reproducibility challenges across different laboratories and research groups, potentially compromising the comparability of findings and the validity of meta-analyses combining multiple datasets [4].

Table 1: Quantitative Comparison of Annotation Methods

Parameter	Manual Annotation	Reference-Based (SingleR)
Processing Time	Hours to days for large datasets	Minutes to hours [7]
Subjectivity	High (expert-dependent)	Low (correlation-based) [5]
Reproducibility	Variable across experts	High and consistent [5]
Scalability	Limited by human effort	Limited only by computing resources [5]
Novel Cell Type Detection	Possible with expert knowledge	Limited to reference types [5]

Reference-Based Annotation with SingleR: Principles and Advantages

Algorithmic Foundation of SingleR

SingleR employs an innovative correlation-based approach that operates independently on each cell in the test dataset [5] [8]. The method begins by calculating Spearman correlation coefficients between the gene expression profile of each single cell and every sample in the reference dataset [6]. This initial analysis utilizes only variable genes present in the reference dataset to maximize biological signal [6]. The resulting multiple correlation coefficients per cell type are then aggregated to generate a single value per cell type per single cell, with SingleR specifically using the 80th percentile of correlation values to prevent misclassification resulting from heterogeneity in the reference samples [6].

The algorithm incorporates a crucial fine-tuning step where the correlation analysis is repeated exclusively for the top cell types identified in the initial phase [6]. This iterative refinement utilizes an optimized set of variable genes specifically selected to distinguish between the most similar cell types, progressively eliminating the lowest-scoring cell type until only two candidates remain [6]. The cell type corresponding to the top value after this final comparison is assigned to the single cell [6]. This sophisticated two-stage approach enables SingleR to achieve high resolution even when distinguishing closely related cell subtypes.

Comparative Advantages Over Manual Approaches

The automated nature of SingleR directly addresses the core limitations of manual annotation. By replacing subjective human judgment with quantitative correlation metrics, SingleR eliminates expert-dependent bias and ensures consistent, reproducible results across different research settings and laboratory environments [5]. The method's computational efficiency enables rapid annotation of datasets comprising hundreds of thousands of cells, effectively removing the scalability constraints that plague manual approaches [5] [7]. This efficiency gain becomes increasingly significant as single-cell technologies continue to evolve toward higher throughput capacities.

Unlike manual methods that rely on prior knowledge of a limited set of marker genes, SingleR leverages the comprehensive transcriptional profiles available in well-curated reference datasets, potentially capturing subtle discriminatory patterns that might escape even expert notice [5] [8]. The method's fine-tuning mechanism specifically enhances its ability to resolve challenging cases where cell types share similar expression patterns for most genes but differ in a small subset of discriminative markers [8] [6].

Diagram Title: SingleR Annotation Workflow

Experimental Protocols for SingleR Implementation

Reference Dataset Selection and Preparation

Principle: The accuracy of SingleR annotation critically depends on selecting an appropriate reference dataset that comprehensively represents the cell types likely present in the test data [8]. The reference must contain high-quality annotations and be generated using compatible technology platforms.

Protocol Steps:

• Reference Selection: Choose a reference dataset matching the species and tissue type of your test data. For human samples, the Blueprint Epigenomics (144 RNA-seq pure immune samples annotated to 28 cell types) and Encode (115 RNA-seq pure stroma and immune samples annotated to 17 cell types) datasets are commonly used [6]. For mouse samples, the Immunological Genome Project (ImmGen) database (830 microarray samples classified to 20 main cell types and 253 subtypes) provides comprehensive coverage [8] [6].

• Data Access: Reference datasets can be accessed through the celldex package in Bioconductor. Load the appropriate reference using dedicated functions (e.g., ImmGenData() for ImmGen reference) [8].

• Quality Assessment: Verify reference quality by examining the distribution of labels and ensuring adequate representation of expected cell types. Check for batch effects and technical artifacts that might compromise annotation accuracy.

• Gene Identifier Matching: Ensure consistent gene annotation between reference and test datasets. When using ImmGen reference with mouse data, set ensembl=TRUE to match the reference's gene annotation with that in the single-cell experiment object [8].

Troubleshooting Tips:

• If annotation accuracy is low, consider trying an alternative reference dataset or combining multiple references.
• For cross-technology comparisons (e.g., Smart-seq2 test data vs. UMI-based references), adjust normalization strategies accordingly [8].

SingleR Execution and Result Interpretation

Principle: SingleR compares gene expression profiles between test and reference datasets through correlation analysis followed by iterative fine-tuning to assign cell type labels [8] [6].

Protocol Steps:

• Data Preprocessing: Normalize both test and reference datasets using appropriate transformations. For the reference dataset, the assay matrix must contain log-transformed normalized expression values [8]. For the test data, raw counts are acceptable as SingleR computes Spearman correlations, which are unaffected by monotonic transformations [8].

• SingleR Execution: Run the core SingleR algorithm with default parameters initially:

• Result Examination: Inspect the returned DataFrame containing prediction results:

  • pred$labels: Vector of predicted labels for each cell
  • pred$scores: Matrix of correlation scores for each cell-label pair
  • pred$delta.next: Difference between best and second-best scores
  • pred$pruned.labels: Labels after pruning of low-confidence assignments [9]

• Quality Control: Implement diagnostic checks to identify low-confidence assignments:

  • Plot score distributions with plotScoreHeatmap(pred)
  • Examine delta values with plotDeltaDistribution(pred)
  • Prune unreliable assignments using pruneScores(pred) [9]

Troubleshooting Tips:

• If fine-tuning takes excessively long, consider reducing fine_tune_times parameter [7].
• For large datasets, use GPU acceleration (method='rapids') to significantly reduce computation time [7].

Validation Using Marker Gene Expression

Principle: Independent validation of SingleR annotations through examination of canonical marker gene expression provides confidence in assignment accuracy and identifies potential misclassifications [9] [3].

Protocol Steps:

• Marker Gene Identification: Extract the marker genes used by SingleR for each label from the metadata() of the SingleR output [9].

• Expression Visualization: Create diagnostic heatmaps showing expression of key marker genes across predicted cell types:

• Cross-Validation: Compare SingleR assignments with unsupervised clustering results to identify discrepancies that may indicate novel cell types or annotation errors [9].

• Credibility Assessment: Apply objective evaluation criteria where an annotation is deemed reliable if more than four marker genes are expressed in at least 80% of cells within the cluster [3].

Troubleshooting Tips:

• If identified markers lack biological meaningfulness or show inconsistent expression, treat corresponding assignments with skepticism [9].
• For closely related cell types with overlapping markers, focus the analysis on the most discriminatory genes identified during fine-tuning [6].

Table 2: SingleR Diagnostic Metrics and Interpretation

Diagnostic Metric	Purpose	Interpretation Guidelines
Correlation Scores	Pre-tuning similarity measures	Higher scores indicate stronger matches; examine spread across labels [9]
Delta Values	Confidence in assignment	Large deltas indicate unambiguous assignments; small deltas suggest uncertainty [9]
Pruned Labels	Automated quality filtering	NA values indicate low-confidence assignments that failed pruning criteria [9]
Marker Expression	Biological plausibility check	Strong expression of label-specific markers validates assignments [9] [3]

Table 3: Key Research Reagents and Computational Resources for SingleR Annotation

Resource	Type	Function	Example Sources
Reference Datasets	Data	Provide annotated transcriptomic profiles for correlation-based matching	Blueprint Epigenomics, ImmGen, Human Cell Atlas [8] [6]
Marker Gene Databases	Knowledge Base	Supply prior knowledge for validation and manual curation	singleCellBase, CellMarker, PanglaoDB [4] [1]
SingleR Software	Tool	Automated cell type annotation algorithm	Bioconductor SingleR package [5] [8]
celldex Package	Resource	Standardized reference datasets for annotation	Bioconductor [8]
Normalization Tools	Computational Method	Prepare expression data for correlation analysis	Seurat, Scanpy [1] [7]

The limitations of manual cell annotation—particularly its inherent subjectivity and poor scalability—present significant challenges in the era of large-scale single-cell genomics. Reference-based automated methods like SingleR offer a robust solution by providing objective, reproducible, and scalable annotation that maintains biological accuracy. The protocols detailed in this Application Note provide researchers with a comprehensive framework for implementing SingleR in diverse experimental contexts, from basic tissue mapping to disease biomarker discovery.

Future methodological developments will likely focus on hybrid approaches that combine the strengths of reference-based and marker-based methods [1], enhanced by artificial intelligence techniques including large language models [3]. Tools like ScInfeR, which integrates information from both scRNA-seq references and marker sets within a graph-based framework, represent the next generation of annotation algorithms that further improve accuracy across diverse sequencing technologies [1]. As the single-cell field continues to evolve toward multi-omic assays and spatial transcriptomics, robust, scalable annotation methods will remain essential for extracting meaningful biological insights from increasingly complex datasets.

Diagram Title: Evolution of Cell Annotation Methods

SingleR represents a transformative approach in single-cell RNA sequencing (scRNA-seq) analysis by implementing an automated, reference-based annotation system that eliminates much of the subjectivity inherent in manual cell type identification. This method operates on a fundamentally simple yet powerful premise: given a reference dataset of samples (either single-cell or bulk) with expertly curated labels, it can transfer these biological annotations to new cells from a test dataset based on similarity in their expression profiles [10]. The methodology effectively leverages existing biological knowledge embedded in reference datasets, allowing researchers to propagate carefully defined cellular identities across experiments in a standardized, reproducible manner [10] [11].

The fundamental advantage of SingleR lies in its ability to bypass the cumbersome process of manually interpreting clusters and defining marker genes for each new dataset—a process that typically requires substantial domain expertise and can introduce considerable inter-observer variability [11]. Instead, with SingleR, this intensive manual work only needs to be performed once during the creation of high-quality reference datasets, after which this annotation framework can be automatically applied to numerous future studies [10]. This approach significantly accelerates analysis workflows while simultaneously improving annotation consistency across laboratories and research projects, making it particularly valuable in large-scale collaborative efforts and in drug development pipelines where standardization is critical.

Core Methodology and Algorithmic Framework

The SingleR Classification Engine

At its computational core, SingleR operates as a robust variant of nearest-neighbors classification, enhanced with specialized tweaks to improve resolution between closely related cell types [10]. The algorithm processes each test cell through a multi-stage procedure that quantifies similarity to reference samples:

• Correlation Calculation: For each test cell, SingleR computes the Spearman correlation between its expression profile and every reference sample [10] [6]. This correlation analysis is performed exclusively on the union of marker genes identified through pairwise comparisons between all labels in the reference data, thereby focusing on features with maximal discriminatory power [10] [8].

• Score Aggregation: The algorithm defines a per-label score as a fixed quantile (default: 0.8) of the correlations across all reference samples bearing that label [10] [6]. This approach effectively mitigates issues arising from heterogeneous reference populations and imbalances in sample numbers across different cell types [10].

• Label Assignment: After repeating this score calculation for all labels in the reference dataset, the label with the highest score becomes SingleR's initial prediction for the test cell [10].

• Fine-Tuning: An optional iterative refinement step improves discrimination between closely related labels by progressively subsetting the reference to only include labels with scores near the maximum and recomputing scores using increasingly specific marker genes [10] [6].

Marker Detection Strategies

SingleR incorporates multiple approaches for identifying the discriminatory genes that power its classification engine:

• Classic Mode: The original implementation identifies marker genes based on the largest positive differences in per-label median log-expression values between label pairs [8]. The number of genes selected from each pairwise comparison follows the formula $500(\frac{2}{3})^{\log_{2}(n)}$, where $n$ represents the number of unique labels in the reference, thereby scaling marker selection complexity with label diversity [8].

• Alternative Methods: For single-cell references where the classic approach may be suboptimal due to data sparsity, SingleR supports alternative marker detection schemes including Wilcoxon rank sum tests, which better accommodate the statistical characteristics of single-cell data [12].

Table 1: Key Algorithmic Parameters in SingleR's Classification Pipeline

Parameter	Default Setting	Function	Impact on Results
Correlation method	Spearman	Measures expression profile similarity	Robust to batch effects; monotonic relationship focused
Score quantile	0.8 (80th percentile)	Aggregates correlations per label	Reduces sensitivity to label heterogeneity
Fine-tuning threshold	0.05	Determines which labels enter iterative refinement	Balances resolution versus computation time
Marker detection	Classic (log-fold change)	Identifies discriminatory genes	Affects feature selection and subtype resolution

Experimental Protocols and Implementation

The following protocol outlines the standard procedure for annotating scRNA-seq data using SingleR with pre-existing reference datasets:

Step 1: Environment Preparation

Step 2: Reference Dataset Acquisition

Step 3: Test Dataset Processing

Step 4: Annotation Execution

Step 5: Result Interpretation and Validation

Advanced Protocol: Single-Cell to Single-Cell Annotation

For researchers working with single-cell reference datasets, the following specialized protocol typically yields superior performance:

Step 1: Reference Single-Cell Data Curation

Step 2: Test Dataset Preparation with Quality Control

Step 3: Specialized Single-Cell Annotation

Step 4: Annotation Diagnostics and Refinement

SingleR Automated Classification Workflow: This diagram illustrates the sequential processing stages within the SingleR algorithm, from initial correlation analysis to final annotation output.

Table 2: Key Reference Datasets and Software Resources for SingleR Implementation

Resource	Type	Description	Application Context
Human Primary Cell Atlas (HPCA)	Microarray reference	713 samples across 37 main cell types [10]	General human cell type annotation
Immunological Genome Project (ImmGen)	Microarray reference	830 mouse immune samples with fine resolution [8]	Mouse immunology studies
Blueprint/Encode	RNA-seq reference	259 human immune and stroma samples [6]	Human hematopoiesis and immunology
celldex package	Data repository	Curated collection of reference datasets [8]	Streamlined reference access
SingleR package	R software	Core algorithm implementation [10]	Primary annotation engine
scRNAseq package	Data repository	Example test datasets for method validation [8] [12]	Protocol development and training

Critical Computational Considerations

Successful implementation of SingleR requires attention to several technical aspects that significantly impact annotation accuracy:

Data Transformation Requirements:

• Reference data must contain log-transformed normalized expression values [8]
• Test data can be provided as raw counts or log-transformed values [8]
• Full-length sequencing technologies (e.g., Smart-seq2) may require TPM transformation for optimal performance with UMI-optimized references [8]

Reference Selection Criteria:

• Reference must contain a superset of expected cell types in test data [8]
• Technology compatibility between reference and test data improves accuracy [8]
• Larger references with more samples per label generally enhance performance [10]

Advanced Applications and Diagnostic Framework

Quality Control and Annotation Validation

SingleR provides built-in diagnostic capabilities to assess annotation confidence and identify potentially problematic assignments:

Score Distribution Analysis:

Delta Score Pruning:

Batch Effect Investigation:

Integration with Experimental Design

The SingleR framework accommodates various experimental designs through parameter optimization:

Table 3: Parameter Optimization Guide for Different Experimental Conditions

Experimental Scenario	Recommended Parameters	Rationale	Expected Outcome
Large datasets (>10,000 cells)	fine.tune=FALSE, subsetting	Computational efficiency	Faster processing with minimal accuracy loss
Closely related cell types	fine.tune=TRUE, increased markers	Enhanced resolution	Better discrimination of similar populations
Cross-technology annotation	de.method="wilcox", TPM transformation	Platform effect mitigation	Improved cross-platform compatibility
Noisy or low-quality data	Increased pruning stringency	False positive reduction	More conservative but reliable annotations

SingleR System Architecture: This diagram illustrates the relationship between core algorithmic components and auxiliary functions within the SingleR ecosystem.

SingleR represents a robust, scalable solution for automated cell type annotation that effectively transfers biological knowledge from carefully curated reference datasets to new experimental data. Its reference-based framework addresses critical challenges in single-cell genomics including reproducibility, standardization, and analytical efficiency. The method's compatibility with diverse reference types—from bulk microarray data to single-cell RNA-seq datasets—and its flexible parameterization make it adaptable to various research contexts from basic biological investigation to pharmaceutical development pipelines.

As single-cell technologies continue to evolve, generating increasingly complex and multimodal datasets, reference-based annotation approaches like SingleR will play an essential role in extracting biologically meaningful insights from these data-rich resources. Future developments will likely focus on integrating additional molecular modalities, improving discrimination of rare cell states, and developing more sophisticated reference composition strategies to address the expanding complexity of cellular taxonomy.

Automated cell type annotation, or label transfer, represents a paradigm shift in the analysis of single-cell RNA sequencing (scRNA-seq) data. This approach aligns with the single-cell field's equivalent to genome aligners, providing a standardized methodology that circumvents the labor-intensive, expert-dependent, and non-scalable nature of manual cluster annotation [5]. Reference-based methods fundamentally operate by comparing cells in a new target dataset against meticulously curated reference profiles of known cell types, assigning each cell to the reference type that its expression profile most closely resembles [5]. SingleR stands as a prominent implementation of this approach, utilizing a correlation-based framework to transfer labels from well-annotated reference datasets to novel target data [5] [1].

The integration of curated biological knowledge into this process significantly enhances its robustness. Curated references encapsulate domain expertise and validated cell type signatures, providing a stable, biologically-grounded foundation for annotation that minimizes technical artifacts and batch effects. This methodology contrasts with exclusively marker-based approaches, which often struggle with closely related cell subtypes due to overlapping marker genes [1]. By leveraging comprehensive reference datasets, tools like SingleR enable researchers to rapidly assign cell identities with confidence, accelerating downstream biological interpretation and discovery.

Quantitative Benchmarking of Annotation Tools

Rigorous performance evaluation is essential for selecting an appropriate cell annotation tool. Benchmarking studies typically assess accuracy, sensitivity, robustness to batch effects, and computational efficiency across diverse biological contexts.

Table 1: Performance Benchmarking of Cell Annotation Tools Across scRNA-seq Datasets

Tool	Methodology	Reported Accuracy	Key Strength	Noted Limitation
SingleR [5] [1]	Reference-based (Spearman correlation)	High (Established baseline)	Speed, simplicity, well-established	Dependent on reference quality and completeness
ScInfeR [1]	Hybrid (Reference + Marker graph)	Superior in benchmarking	Robustness to batch effects; versatile across technologies (scRNA-seq, scATAC-seq, spatial)	-
scExtract [13]	LLM-based (Article text + data)	Higher than established methods	Automates processing and annotation using article context; enables prior-informed integration	-
LICT [3]	Multi-LLM integration	High consistency with experts; superior efficiency/accuracy	Objective credibility evaluation; reference-free; handles multifaceted cell populations	Performance dips in low-heterogeneity datasets

The benchmarking reveals that hybrid methods, which integrate multiple sources of biological knowledge, tend to outperform single-modality approaches. For instance, ScInfeR's hybrid framework, which combines information from both scRNA-seq references and marker sets, demonstrated superior performance in over 100 cell-type prediction tasks across multiple atlas-scale scRNA-seq, scATAC-seq, and spatial datasets [1]. Similarly, the LLM-based tool scExtract was validated to achieve higher accuracy than established methods like SingleR, scType, and CellTypist across multiple human tissues [13]. A critical finding is that the performance of any individual method can be context-dependent. For example, LLM-based annotations show diminished performance in low-heterogeneity datasets where transcriptional differences are subtler [3]. This underscores the advantage of tools that incorporate iterative validation or multi-model integration to mitigate such weaknesses.

Protocol for Cell Annotation with SingleR and Enhanced Workflows

The following section provides a detailed, practical protocol for performing cell type annotation using the core SingleR method, along with strategies for integrating additional curated knowledge to enhance accuracy.

SingleR Core Annotation Protocol

Primary Materials & Reagents:

• Computational Environment: R (version 4.5.1 or higher) with Bioconductor.
• Software Packages: SingleR package (v1.20.0+) [5], Seurat package for single-cell data handling [1].
• Reference Data: A high-quality, well-annotated scRNA-seq dataset relevant to the biological system of interest (e.g., from the Human Cell Atlas, Tabula Sapiens, or cellxgene [1] [13]).

Step-by-Step Methodology:

• Data Preprocessing: Begin by processing both the target (unannotated) and reference datasets. This includes standard quality control (filtering cells by mitochondrial gene percentage and library size), normalization, and log-transformation. The reference dataset must be a normalized expression matrix with pre-assigned cell type labels.

• Reference Selection and Curation: This is a critical step for unbiased results. Select a reference dataset that comprehensively represents the expected cell types in your target data. If a single reference is insufficient, consider combining multiple references, ensuring compatibility and batch correction. The quality of the annotation is directly dependent on the quality and relevance of the reference [5] [1].

• Label Transfer with SingleR: Execute the core SingleR function. The basic command in R is:

  SingleR performs a Spearman correlation for each cell in the target dataset against every cell in the reference, assigning the label of the best-matching reference cell [5] [1].

• Result Interpretation and Diagnostics: SingleR provides diagnostic scores (e.g., per-cell tuning scores) to assess the confidence of each label assignment. Visually inspect these scores and consider filtering out low-confidence assignments before proceeding to downstream analysis.

Workflow for Integrating Curated Marker Knowledge

To leverage curated biological knowledge beyond a single reference, a hybrid workflow incorporating marker genes can be implemented, as inspired by tools like ScInfeR [1].

Diagram 1: Hybrid annotation workflow integrating reference and marker knowledge.

• Parallel Annotation Tracks: In parallel to the SingleR annotation, utilize a curated marker database (e.g., ScInfeRDB, which covers 329 cell-types and 2497 gene markers across 28 human and plant tissues) [1]. Assess the expression of cell-type-specific positive and negative markers in the target dataset.

• Annotation Integration: Compare the results from SingleR and the marker-based analysis. High-confidence labels are those where both methods agree. For discrepant labels, investigate by examining the correlation scores from SingleR and the specificity of marker expression.

• Hierarchical Sub-type Refinement: For broad cell classes (e.g., "T cells"), perform a second round of annotation using a sub-type-specific reference or marker set to resolve finer heterogeneity. This hierarchical approach, inspired by ScInfeR's framework, significantly improves sub-type discrimination [1].

Essential Research Reagent Solutions

The following reagents and data resources are fundamental for implementing robust and unbiased cell annotation protocols.

Table 2: Key Reagents and Resources for Cell Annotation

Resource Name	Type	Primary Function in Annotation	Key Features
Tabula Sapiens Atlas [1]	scRNA-seq Reference Data	Provides a comprehensive, high-quality human reference.	Multi-tissue, carefully annotated, serves as a gold-standard benchmark.
ScInfeRDB [1]	Curated Marker Database	Supplies cell-type-specific gene signatures for marker-based validation.	Hierarchical database of 2497 markers for 329 cell-types across 28 tissues.
cellxgene [13]	Data Platform / Curated Corpus	Source of pre-processed, annotated public datasets for use as references.	Largest literature-curated single-cell database (1458+ datasets).
SingleR Bioconductor Package [5]	Software Tool	Executes the core reference-based label transfer algorithm.	R-based, integrates with Bioconductor analysis workflows, fast correlation-based method.
Peripheral Blood Mononuclear Cell (PBMC) Data [1] [3]	Benchmarking Dataset	Serves as a standard for initial tool validation and benchmarking.	Well-characterized, highly heterogeneous, widely used for evaluation.

Leveraging curated biological knowledge through reference-based annotation with tools like SingleR provides a powerful, scalable, and less biased alternative to manual cell typing. The key advantages of this paradigm are its foundation in established biological data, which promotes reproducibility and standardization across studies. As the field evolves, the integration of multiple knowledge sources—including reference datasets, curated marker genes, and even textual information from scientific articles via LLMs—is proving to be a superior strategy. This hybrid approach, exemplified by next-generation tools like ScInfeR and scExtract, enhances accuracy, robustness to batch effects, and enables the reliable identification of both common and rare cell types, ultimately accelerating discovery in biomedical research and drug development.

SingleR is a powerful computational method for the unbiased cell type recognition of single-cell RNA sequencing (scRNA-seq) data. It functions as a robust variant of nearest-neighbor classification, leveraging existing reference transcriptomic datasets with known labels to automatically annotate cell types in a new test dataset [10]. This process transfers biological knowledge from well-characterized references to new experiments, eliminating the need for manual cluster interpretation and marker gene definition for every new dataset [10]. The core of SingleR's algorithm involves calculating the Spearman correlation between the expression profile of each test cell and every reference sample. It then assigns the label with the highest score, optionally employing an iterative fine-tuning step to improve resolution between closely related cell types [10]. The success and accuracy of this method hinge entirely on two critical inputs: the properly formatted test dataset and a carefully chosen reference dataset. The following sections provide a detailed protocol for preparing these essential inputs, enabling researchers to effectively harness SingleR for cell annotation in biomedical research and drug development.

Essential Input 1: Your Single-Cell Test Dataset

The test dataset, which is the subject of your annotation experiment, must be formatted correctly and undergo appropriate quality control to ensure reliable results from SingleR.

Data Format and Object Types

SingleR is designed for flexibility, accepting test data in several common formats. A numeric matrix is the most straightforward format, where rows represent genes and columns represent cells [14]. Alternatively, SingleR can directly work with objects from popular single-cell analysis frameworks, notably the SingleCellExperiment object [8] or the Seurat object [14]. Using these objects can streamline the workflow, as they seamlessly integrate with other analysis steps. When extracting data from a Seurat object, you can provide either raw counts or normalized counts. The raw counts are stored in the 'counts' layer, while the normalized counts are stored in the 'data' layer [14].

Data Preprocessing and Requirements

A key advantage of SingleR is its minimal preprocessing requirements for the test data. The algorithm computes Spearman correlations within each cell, a metric that is unaffected by monotonic transformations like log-transformation or cell-specific scaling. Consequently, it is perfectly acceptable to provide the raw counts for the test dataset [8]. However, an important exception arises when comparing data from full-length sequencing technologies (e.g., Smart-seq2) to references designed for unique molecular identifier (UMI) protocols. In such cases, processing the test counts to transcripts-per-million (TPM) values is recommended for better performance, as UMI-based references are less sensitive to gene length differences [8]. While not always mandatory, normalization steps like log1p() and normalize_total() are often applied in practice to improve outcomes [7].

Critical Quality Control Steps

Annotation with SingleR is performed independently on each cell, making it orthogonal to quality control (QC). However, low-quality cells lack the information needed for accurate assignment, and their removal is crucial for interpreting the final results [8]. The annotation results can be filtered post-analysis based on QC metrics without needing to re-run SingleR [8]. Standard cell QC metrics should be examined to remove damaged cells, dying cells, and doublets. The three primary metrics are [15]:

• Total UMI count (count depth): Low counts may indicate damaged cells.
• Number of detected genes: Low numbers suggest damaged or low-quality cells.
• Fraction of mitochondrial counts: A high proportion is indicative of dying cells.

Table 1: Key Quality Control Metrics for Single-Cell Test Data

Metric	Description	Indicator of Problem	Suggested Action
Total UMI Count	The total number of transcripts detected per cell.	Low counts indicate damaged or empty droplets.	Filter out cells below a threshold (e.g., 500).
Number of Genes	The number of unique genes detected per cell.	Low numbers indicate damaged cells; very high numbers may indicate doublets.	Filter based on lower and upper thresholds.
Mitochondrial Fraction	The percentage of transcripts derived from mitochondrial genes.	High fraction indicates apoptotic or dying cells.	Filter out cells exceeding a threshold (e.g., 10-20%).

Essential Input 2: The Reference Dataset

The choice of reference dataset is arguably the most critical decision in the annotation workflow, as it directly determines the possible labels that SingleR can assign.

Reference Data Requirements

The reference dataset must be a normalized matrix of expression values. Specifically, the assay matrix must contain log-transformed normalized expression values [8]. This requirement exists because the default marker detection scheme in SingleR's classic mode computes log-fold changes by subtracting the medians of expression values, an operation that is only meaningful on a log-transformed scale [8]. Furthermore, the reference must have a set of labels assigned to each sample or cell. These labels can vary in resolution, with some references providing broad cell categories (label.main) and others offering more detailed subtypes (label.fine) [8] [14].

Selecting an Appropriate Reference

The guiding principle for reference selection is to choose a reference that contains a superset of the labels you expect to be present in your test dataset [8]. Using a reference that lacks the cell types in your sample will lead to incorrect or poor-quality annotations. Therefore, the biological context is paramount. For a study on human peripheral blood mononuclear cells (PBMCs), a human immune-specific reference like the Database of Immune Cell Expression (DICE) is more appropriate than a broad reference that includes non-immune cell types from solid tissues [14]. Whenever possible, using a reference generated from a similar technology or protocol as the test dataset can also minimize batch effects and improve accuracy [8].

Curated Reference Datasets

The celldex R package provides easy access to several expertly curated reference datasets, saving researchers the effort of building their own. These datasets are derived from bulk RNA-seq or scRNA-seq experiments and cover both human and mouse model systems.

Table 2: Commonly Used Reference Datasets Available in the celldex Package

Reference Name	Species	Description	Key Cell Types	Label Granularity
Human Primary Cell Atlas (HPCA) [16]	Human	A comprehensive reference derived from a wide range of pure human primary cell types.	Immune cells, stem cells, stromal cells, and more.	Broad (main) and fine-grained (fine).
BlueprintEncodeData [7]	Human	Integrates data from the Blueprint and ENCODE projects, focusing on hematopoietic cell types.	Immune and progenitor cells from blood and bone marrow.	Broad and fine-grained.
MonacoImmuneData [17]	Human	A reference of pure immune cell types from the study by Monaco et al.	Detailed immune cell subsets (e.g., T cell, B cell, monocyte subtypes).	Fine-grained.
MouseRNAseqData [7]	Mouse	A reference dataset derived from pure cell types of mouse origin.	A wide array of mouse cell types from various tissues.	Broad and fine-grained.
ImmGenData [8]	Mouse	From the Immunological Genome Project, offering a deep resource for mouse immune cells.	Highly detailed immune cell types and stages of differentiation.	Very fine-grained.

Using Custom Reference Datasets

While curated references are convenient, SingleR also supports user-supplied reference datasets. This is essential for annotating cell types not covered in public resources or for using internal, proprietary data. A custom reference can be supplied as long as it is formatted as a SummarizedExperiment object (or similar) containing a matrix of log-expression values and a vector of labels for each reference sample [8]. This allows for incredible flexibility, enabling researchers to create bespoke references tailored to specific tissues, diseases, or experimental conditions.

Integrated Protocol: A Step-by-Step Workflow for SingleR Annotation

This protocol integrates the preparation of both test and reference data into a complete workflow for cell type annotation with SingleR.

The diagram below illustrates the logical flow of a complete SingleR analysis, from data input to final annotation.

Step-by-Step Procedure

• Prepare the Test Data

  • Load your test single-cell data (e.g., from a 10X Genomics output) into R and create a SingleCellExperiment or Seurat object [8] [14].
  • Perform rigorous quality control. Calculate QC metrics and filter out cells with low UMI counts, low gene counts, or an abnormally high fraction of mitochondrial counts [15]. The specific thresholds are experiment-dependent but are critical for obtaining a clean result.
  • While SingleR can use raw counts, apply logNormalize or similar transformations if you are using a Seurat-based workflow for downstream analysis beyond SingleR.

• Acquire and Prepare the Reference Data

  • Install and load the celldex package. Select the most appropriate reference for your sample's biological context and species [14]. For example, for human PBMCs, HumanPrimaryCellAtlasData or MonacoImmuneData are suitable starting points.
  • Download the reference dataset (e.g., ref <- celldex::HumanPrimaryCellAtlasData()).
  • Examine the available labels using unique(ref$label.main) and unique(ref$label.fine) to understand the annotation granularity [14].

• Execute SingleR

  • Run the core SingleR function, specifying the test data, the reference data, and the column containing the reference labels.

  • The function returns a DataFrame object where each row corresponds to a cell in the test data, containing the predicted labels, confidence scores, and other diagnostic information [8].

• Interpret and Integrate Results

  • Examine the distribution of predicted labels using table(pred$labels). Cross-reference these results with your prior biological knowledge to assess their plausibility [8].
  • SingleR provides a pruned.labels column where low-confidence assignments are replaced with NA. Pay attention to these pruned labels.
  • Finally, transfer the confident labels (either the first labels or the pruned labels) back into your original Seurat or SingleCellExperiment object for downstream analysis and visualization [14].

Table 3: Key Research Reagent Solutions for SingleR Annotation

Item	Function / Description	Example / Source
SingleR R Package	The core software for performing reference-based cell type annotation.	Bioconductor (https://bioconductor.org/packages/SingleR/) [17]
celldex R Package	Provides a curated collection of reference datasets for both human and mouse studies.	Bioconductor (https://bioconductor.org/packages/celldex/) [8]
Seurat	A comprehensive toolkit for single-cell genomics data preprocessing, analysis, and visualization.	CRAN / Satija Lab (https://satijalab.org/seurat/) [14]
SingleCellExperiment	A S4 class for storing and manipulating single-cell genomics data, used as an input by many Bioconductor packages.	Bioconductor [8]
scRNA-seq Reference Datasets	Pre-formatted, log-normalized expression matrices with expert cell type labels.	HumanPrimaryCellAtlasData(), BlueprintEncodeData(), MonacoImmuneData() from celldex [16] [7] [17]
High-Performance Computing (HPC) Resources	Essential for processing large scRNA-seq datasets, as SingleR calculations can be computationally intensive.	Institutional HPC clusters or cloud computing services [7]

Troubleshooting and Best Practices

• Unexpected Annotations: If SingleR returns implausible cell type labels, the most likely cause is a mismatch between your test data and the reference. Verify that your test sample's biological origin is well-represented in the reference. For example, a sorted hematopoietic stem cell (HSC) population showing many differentiated cell types may indicate contamination in the sample or an inappropriate reference [8].
• Computational Time: SingleR's fine-tuning process can be time-consuming for very large datasets (e.g., tens of thousands of cells). If this is prohibitive, consider running SingleR on subsets of the data and combining the results, or using clustering information to speed up the calculation by annotating at the cluster level [7] [18].
• Reference Choice is Key: The performance of SingleR is heavily dependent on the quality and relevance of the reference dataset. Invest time in selecting the best possible reference, and do not hesitate to try multiple references to see which yields the most biologically coherent results [8] [14]. The availability of curated references through celldex significantly lowers the barrier to this critical step.

In the evolving landscape of single-cell RNA sequencing (scRNA-seq) analysis, accurate cell type identification remains a fundamental challenge. SingleR is an automated computational method that addresses this challenge by leveraging well-characterized reference datasets to annotate cell types in new, unlabeled test data [10]. This approach transforms biological knowledge embedded in reference datasets into transferable classification schemes, eliminating the need for manual cluster interpretation and marker gene selection with each new dataset [10] [19]. The method functions as a robust variant of nearest-neighbor classification, employing correlation-based scoring and iterative fine-tuning to achieve precise label assignments [10] [20]. For drug development professionals, SingleR offers a standardized framework for cell type identification across disease models, clinical samples, and preclinical studies, enabling more consistent biomarker identification and patient stratification strategies [21].

The core algorithm operates on a simple but powerful principle: for each cell in a test dataset, SingleR identifies the most similar reference samples based on gene expression patterns and assigns the corresponding label [19]. This process transfers biological knowledge from expertly annotated references to new datasets, creating a powerful tool for propagating cell type annotations across studies, experimental platforms, and laboratories [10]. As single-cell technologies increasingly transform drug discovery and development—from target identification to understanding drug mechanisms of action—reliable automated annotation methods like SingleR become essential infrastructure for extracting meaningful biological insights from complex cellular heterogeneity [21].

Core Algorithm Mechanics: From Correlation to Classification

Spearman Correlation as the Foundation

The SingleR algorithm employs Spearman's rank correlation coefficient as its primary similarity metric, calculating this measure between each test cell's expression profile and every reference sample [10] [20] [22]. This correlation is computed exclusively using the union of marker genes identified through pairwise comparisons between all labels in the reference data, thereby focusing on features with maximal discriminatory power [10] [8]. The selection of Spearman correlation provides distinct advantages for scRNA-seq data analysis, including reduced sensitivity to technical batch effects and outlier values that commonly plague sequencing experiments [23]. As a non-parametric rank-based method, it captures monotonic relationships without assuming normal data distribution, making it particularly suitable for count-based sequencing data where expression values may not follow Gaussian assumptions [23] [22].

The correlation calculation process involves systematic comparison between test cells and reference samples. For each test cell, the algorithm computes its correlation with all reference samples, then aggregates these correlations by reference label [10]. Rather than using simple averages that could be biased by label size heterogeneity, SingleR defines a per-label score as a fixed quantile (default: 0.8) of the correlation distribution across all samples with that label [10] [20]. This approach ensures that labels with different numbers of reference samples are compared fairly and prevents penalization of heterogeneous cell types by only requiring strong similarity to a subset of reference samples [10]. The label with the highest aggregated score becomes the preliminary assignment for the test cell [10].

Scoring and Initial Label Assignment

The scoring mechanism in SingleR incorporates sophisticated statistical handling to ensure robust classification across diverse cellular populations. The quantile-based scoring system (default 80th percentile) effectively captures the characteristic expression pattern of each label while mitigating the influence of outlier reference samples [10] [20]. This strategy proves particularly valuable when dealing with cellular states that exhibit continuous transitions or when reference labels contain internal heterogeneity, as it only requires that a test cell strongly resembles a substantial subset—but not necessarily all—of a label's reference profiles [10].

Following score calculation for all reference labels, each test cell receives an initial assignment corresponding to the label with the highest score [19] [22]. This initial assignment represents the starting point for the refinement process that follows. The entire process—from correlation calculation to initial labeling—focuses on genes with the strongest discriminatory power, as determined by precomputed marker genes for each label [8]. These marker genes are identified through systematic pairwise comparisons between all labels in the reference, ensuring the selected feature set contains genes that distinguish each label from any other [8].

Figure 1: SingleR Core Scoring Workflow. The algorithm computes Spearman correlations between each test cell and all reference samples, then calculates per-label scores as a fixed quantile of these correlations before assigning the label with the highest score.

The Fine-Tuning Process: Resolving Ambiguity in Cell Identity

Iterative Refinement of Label Assignments

The fine-tuning step represents SingleR's sophisticated mechanism for resolving classification ambiguity between closely related cell types [10] [24]. This process initiates by identifying labels with scores falling within a narrow threshold of the top score (determined by the fine.tune.thres parameter) [24]. The algorithm then subsets the reference dataset to include only these top candidate labels and recalculates scores using a refined marker gene set specifically tailored to distinguish between these remaining options [10] [20]. By focusing exclusively on markers relevant to the most plausible labels, fine-tuning significantly enhances resolution for distinguishing biologically similar cell states that might be confused in the initial broad classification [8].

This refinement process operates iteratively, with each round further narrowing the candidate label set until only one label remains [24]. At each iteration, the algorithm identifies variable genes within the reference dataset specifically for the remaining labels and recalculates correlation scores using only these discriminatory features [24]. The progressive focusing on increasingly specific marker genes enables SingleR to distinguish subtle transcriptional differences between closely related cell types, such as different functional states within the same lineage or maturation stages of developing cells [10]. This capability proves particularly valuable in drug development contexts where understanding subtle shifts in cellular states in response to treatment can reveal important mechanisms of action [21].

Technical Implementation of Fine-Tuning

The fine-tuning function in SingleR incorporates several customizable parameters that control the precision of the refinement process. The fine.tune.thres parameter establishes the score range below the maximum for including labels in fine-tuning—a smaller threshold creates a more exclusive candidate set, while a larger value permits more labels into the refinement process [24] [25]. The quantile.use parameter determines how correlation coefficients are aggregated across reference samples for each label, with the default value of 0.8 providing robustness against outlier references [24]. For marker gene selection during fine-tuning, users can employ either standard deviation-based thresholds (sd.thres) or differential expression methods (genes="de") to identify the most informative features [24].

From an implementation perspective, the fine-tuning process can be computationally intensive for large datasets [18]. The SingleR package offers performance optimizations, including parallelization through the numCores parameter, to address this challenge [24]. For very large datasets (tens of thousands of cells), the documentation recommends running SingleR on subsets of data and combining results, as the fine-tuning process may become prohibitively slow otherwise [18]. These practical considerations ensure the method remains applicable to the growing scale of single-cell studies in modern drug discovery pipelines, where sample sizes continue to increase with technological advancements [21].

Figure 2: SingleR Fine-Tuning Process. This iterative workflow progressively refines label assignments by focusing on top candidate labels and recomputing scores with increasingly specific marker genes until unambiguous assignment is achieved.

Experimental Protocols for SingleR Implementation

Reference Dataset Selection and Preparation

The foundation of successful SingleR analysis lies in appropriate reference selection and processing. Reference datasets must contain log-transformed normalized expression values, as the default marker detection scheme computes log-fold changes from median expressions [8]. For single-cell references, users should perform standard quality control including removal of low-quality cells and normalization before employing them in SingleR [19] [22]. The reference dataset should encompass a superset of the cell types expected in the test data, with carefully validated labels that represent biological truth [8]. For drug discovery applications, references capturing disease-relevant cell states prove particularly valuable for detecting pathological cellular populations in patient samples [21].

The SingleR ecosystem provides access to curated reference datasets through the celldex package, including the Human Primary Cell Atlas (HPCA), ImmGen, and mouse cell atlases [10] [19]. These resources offer pre-processed references with multiple annotation levels (main labels, fine labels, ontological terms) to support different resolution needs [19]. When preparing custom references, researchers should ensure gene identifiers match between reference and test datasets and consider technology differences between platforms—for instance, when comparing full-length SMART-seq2 data to UMI-based references, TPM normalization may improve cross-technology compatibility [8].

Marker Gene Detection Methods

SingleR provides multiple algorithms for marker gene detection, each with distinct advantages for different reference types. The classic method computes log-fold changes between per-label median expressions and selects genes with the largest positive differences [8]. This approach works efficiently with bulk RNA-seq references or well-replicated single-cell data but may struggle with sparse single-cell matrices where medians are frequently zero [19]. For single-cell references, the Wilcoxon rank sum test offers improved performance by identifying differentially expressed genes without assuming normal distribution, making it more robust to technical zeros and dropouts characteristic of scRNA-seq data [19] [22]. Alternative methods like the Welch t-test accommodate unequal variances between groups, which can occur when comparing cell types with different expression variances [25].

Table 1: Marker Gene Detection Methods in SingleR

Method	Key Mechanism	Best Application Context	Advantages	Limitations
Classic	Log-fold change between medians	Bulk RNA-seq references, well-replicated scRNA-seq	Computational efficiency, intuitive interpretation	Poor performance with sparse data (many zeros)
Wilcoxon Rank Sum Test	Difference in expression ranks	Single-cell references, sparse data	Non-parametric, robust to outliers and zeros	Computationally intensive for large references
Welch t-test	Difference in means with unequal variances	References with heterogeneous variance	Accommodates variance differences between groups	Assumes approximately normal distribution

Quality Control and Diagnostic Procedures

SingleR incorporates multiple diagnostic approaches to evaluate annotation quality. The plotScoreHeatmap() function visualizes scores for all cells across reference labels, enabling researchers to identify confident assignments (single high score) versus uncertain calls (multiple similar scores) [19] [22]. The delta score—representing the difference between the assigned label's score and the median across all labels for each cell—serves as a key confidence metric [25] [19]. The plotDeltaDistribution() function displays these deltas across cells for each label, highlighting assignments with marginal confidence [19].

The pruning process removes low-quality assignments using outlier detection within per-label delta distributions [25]. Cells with delta values falling more than a specified number of median absolute deviations (MADs) below the median are classified as "pruned" and receive NA labels [25]. This approach effectively identifies cells whose true type may be absent from the reference or those with ambiguous expression profiles [19]. For drug development applications, these quality control steps ensure that subsequent analyses—such as identifying cell type-specific drug responses—build upon reliable cellular annotations [21].

Table 2: Key Research Reagent Solutions for SingleR Analysis

Resource Category	Specific Examples	Function in SingleR Workflow	Implementation Considerations
Reference Datasets	Human Primary Cell Atlas (HPCA), ImmGen, Tabula Sapiens, Tabula Muris	Provide annotated expression profiles for cell type recognition	Ensure compatibility with test data species and technology
Software Packages	SingleR (Bioconductor), celldex, scRNAseq, Seurat	Implement annotation algorithms and provide access to reference data	Maintain version consistency for reproducible analysis
Marker Detection Algorithms	Classic, Wilcoxon, Welch t-test	Identify discriminatory genes for cell type classification	Match method to reference data type (bulk vs. single-cell)
Visualization Tools	plotScoreHeatmap(), plotDeltaDistribution()	Diagnose annotation quality and confidence	Interpret patterns to identify misassignment or novel types
Quality Control Metrics	Delta scores, pruning thresholds, fine-tuning parameters	Filter ambiguous assignments and refine predictions	Adjust stringency based on biological complexity

Applications in Drug Discovery and Development

SingleR's automated annotation approach provides particular value in pharmaceutical research, where consistent cell type identification across multiple experiments and model systems enhances reproducibility and translational potential [21]. In target identification, SingleR enables improved disease understanding through precise cell subtyping in patient tissues, revealing pathogenic cellular states that may represent therapeutic targets [21]. For example, studies have applied scRNA-seq to define T-cell states associated with response or resistance to checkpoint inhibitor therapies in melanoma, identifying potential biomarkers for patient stratification [21]. Similarly, cancer cell states uncovered through single-cell analysis have revealed resistance programs associated with T-cell exclusion, suggesting new combination therapy approaches [21].

In preclinical development, SingleR aids the selection of relevant disease models by characterizing their cellular composition relative to human conditions [21]. The method can identify model-specific cell populations absent in human disease, potentially explaining divergent therapeutic responses [21]. Furthermore, SingleR applications in functional genomics screens—where CRISPR perturbations are combined with scRNA-seq reading—enhance target credentialing by revealing cell type-specific effects of gene manipulations [21]. As single-cell technologies continue advancing, reference-based annotation with SingleR will play an increasingly central role in translating cellular heterogeneity insights into improved therapeutic strategies [21].

SingleR represents a sophisticated approach to automated cell type annotation that combines robust correlation metrics with iterative fine-tuning to achieve precise classification. The use of Spearman correlation provides technical resilience to batch effects and data distribution challenges, while the fine-tuning process enables resolution of closely related cellular states. For the drug development community, these capabilities support more standardized cell type identification across studies, enhancing reproducibility and translational potential. As single-cell applications continue expanding in basic research and clinical development, reference-based annotation methods like SingleR will remain essential tools for extracting biological meaning from cellular heterogeneity.

Your Hands-On SingleR Workflow: From Raw Data to Annotated Cells

In reference-based single-cell RNA sequencing (scRNA-seq) analysis, the preparation of data objects is a critical preliminary step that fundamentally determines the success of all subsequent biological interpretations. The quality of cell type annotation using tools like SingleR is inherently dependent on the proper structure and normalization of the input data [5]. Within the broader workflow of single-cell analysis, which encompasses clustering, dimensionality reduction, and differential expression, data preparation forms the essential foundation upon which reliable annotations are built.

The two dominant object structures in the field represent complementary ecosystems: Seurat objects within the R environment and SingleCellExperiment (SCE) objects within the Bioconductor framework [26]. Seurat offers a comprehensive and versatile toolkit supporting a wide range of analytical functionalities, including spatial transcriptomics and multiome data integration [27] [26]. Conversely, the SingleCellExperiment ecosystem provides a robust, standardized base class that ensures interoperability across numerous Bioconductor packages, facilitating sophisticated statistical analyses and method benchmarking [26]. Understanding the construction, manipulation, and interconversion of these data structures is therefore paramount for researchers embarking on reference-based cell annotation with SingleR.

Understanding the Core Data Structures

The Seurat Object Architecture

The Seurat object serves as a centralized container for all single-cell data and associated metadata. Its structure is organized into several key components that work in concert to facilitate a comprehensive analytical workflow:

• Assays: Represent the core expression data, typically stored in separate slots for raw counts (RNA assay), normalized data (SCT assay via sctransform), or integrated data (integrated assay). Each assay contains three main layers: counts (raw data), data (normalized values), and scale.data (scaled values for dimensionality reduction) [27].
• Metadata (meta.data): A data frame storing cell-level information including quality control metrics (e.g., nCount_RNA, nFeature_RNA, percent mitochondrial reads), sample origins, and cluster identities [27].
• Dimensional Reductions: Slots for storing the results of techniques like PCA, UMAP, and t-SNE, which are crucial for visualization and downstream analysis [28].
• Graphs: Contain nearest-neighbor graphs used for clustering and trajectory inference.
• Project Information: Basic information including project name and assay type.

A critical advancement in Seurat v5 is the introduction of the Layers system within assays, which enables more efficient storage and manipulation of multiple versions of the same data (e.g., raw and normalized counts) without requiring separate assays [28]. This architecture is particularly beneficial for integration workflows, where IntegrateLayers() can harmonize data across batches or conditions using methods like CCA, Harmony, or RPCA [28].

The SingleCellExperiment Ecosystem

The SingleCellExperiment (SCE) object provides a standardized foundation for single-cell genomic analyses within the Bioconductor project, offering several specialized components:

• Assays: A list of matrices containing expression values, analogous to Seurat assays, with the primary matrix typically stored as the first element.
• ColData: Column metadata containing cell-level annotations, comparable to Seurat's meta.data, including quality metrics, batch information, and cluster assignments.
• RowData: Row metadata containing gene-level annotations, such as feature types and biological annotations.
• ReducedDims: A list of dimensionality reduction results (e.g., PCA, UMAP, t-SNE).
• AltExps: A container for storing alternative feature sets, such as data for spike-in transcripts or antibody-derived tags (ADT) from CITE-seq.

The SCE ecosystem promotes interoperability through packages like scran for robust normalization, scater for quality control and visualization, and ZINB-WaVE for dimensionality reduction under zero-inflated assumptions [26]. This modular approach facilitates seamless transitions between specialized analytical methods while maintaining data integrity.

Comparative Analysis of Object Structures

Table 1: Comparative Analysis of Seurat and SingleCellExperiment Object Structures

Feature	Seurat Object	SingleCellExperiment Object
Primary Use Case	End-to-end analysis with integrated workflows	Modular, interoperable analysis within Bioconductor
Expression Data Storage	Multiple Assays with counts, data, and scale.data slots	Assay list containing one or more matrices
Cell Metadata	meta.data slot as a data frame	colData slot as a DataFrame
Feature Metadata	Stored within assays	rowData slot as a DataFrame
Dimensionality Reductions	Individual slots (pca, umap, tsne)	reducedDims list container
Multi-Modal Data Support	Integrated assays (e.g., SCT, integrated)	altExps for alternative features
Key Advantage	Comprehensive, all-in-one toolkit	Standardized base for method interoperability

Experimental Protocols for Data Preparation

Comprehensive Workflow for Data Preparation

The transformation of raw single-cell data into analysis-ready objects follows a systematic workflow encompassing quality control, normalization, feature selection, and dimensionality reduction. The diagram below illustrates this comprehensive process:

Protocol 1: Creating and Preparing a Seurat Object

This protocol details the step-by-step process for constructing a properly formatted Seurat object from a count matrix, with specific emphasis on parameter selection for optimal SingleR annotation.

Materials Required:

• Raw or filtered count matrix (genes × cells)
• Cell-level metadata (optional but recommended)
• R environment with Seurat installed

Procedure:

• Object Creation and Quality Control

• Cell Filtering Based on QC Metrics

• Normalization and Variable Feature Selection

• Scaling and Dimensionality Reduction

Technical Notes:

• The min.cells parameter filters out genes detected in fewer than the specified number of cells, reducing noise.
• The min.features parameter removes cells with fewer than the specified number of detected genes, eliminating empty droplets or damaged cells.
• For datasets with significant technical variability, SCTransform provides superior normalization by explicitly modeling the mean-variance relationship [27].
• When regressing out unwanted variation (e.g., mitochondrial percentage), avoid over-correction which might remove biological signal.

Protocol 2: Creating and Preparing a SingleCellExperiment Object

This protocol outlines the creation of a SingleCellExperiment object, leveraging the Bioconductor ecosystem for robust data preprocessing.

Materials Required:

• Count matrix (genes × cells)
• Cell metadata dataframe
• Gene metadata dataframe (optional)
• R environment with SingleCellExperiment and scran packages

Procedure:

• Object Creation and Quality Control

• Cell Filtering and Normalization

• Feature Selection and Dimensionality Reduction

Technical Notes:

• The deconvolution method for normalization in scran accounts for composition biases in highly heterogeneous cell populations.
• modelGeneVar identifies highly variable genes while accounting for the mean-variance relationship, similar to the vst method in Seurat.
• The quickCluster step ensures that size factors are computed within homogeneous cell subgroups, improving normalization accuracy.

Protocol 3: Object Interconversion and Troubleshooting

Interconversion between Seurat and SingleCellExperiment objects enables researchers to leverage the strengths of both ecosystems. However, version compatibility issues can arise, particularly with updates to object structures.

Procedure:

• Converting SingleCellExperiment to Seurat

• Converting Seurat to SingleCellExperiment

Troubleshooting Common Issues:

• Error with layer specification: Recent versions of Seurat have introduced a Layers system that can cause conversion errors if not properly specified [29]. The error "arg' should be one of 'counts', 'data', 'scale.data'" indicates a layer specification issue. Explicitly specify the data layer using thedataparameter rather thanlayer`.

• Metadata preservation: Ensure that cell-level metadata is correctly transferred between objects by verifying column names in colData(sce) and seurat_obj[[]].

• Assay consistency: Confirm that the same normalization method (e.g., logcounts vs. SCT) is used consistently throughout the analysis pipeline.

Research Reagent Solutions for Single-Cell Preparation

Table 2: Essential Research Reagents and Platforms for Single-Cell Data Generation

Reagent/Platform	Primary Function	Compatibility with Data Structures
10X Genomics Chromium	Droplet-based single-cell partitioning and barcoding	Direct input to Cell Ranger, outputs compatible with both Seurat and SCE
Cell Ranger	Processing raw FASTQ files to count matrices	Generates standardized output readable by both Seurat and SingleCellExperiment [26]
TotalSeq Antibodies (BioLegend)	Antibody-derived tags for protein surface marker detection	Supported in Seurat's CITE-seq analysis and SCE's altExps [30]
scRNA-seq Platform Kits	Library preparation for various chemistries (3', 5', full-length)	Processed data compatible with both object types with appropriate normalization

Integration with SingleR Annotation Workflow

Proper data preparation directly influences the performance of SingleR annotation. The following diagram illustrates how prepared objects interface with the SingleR ecosystem:

The prepared Seurat or SingleCellExperiment object serves as the essential input for SingleR, which compares cells in the test dataset against curated reference profiles of known cell types [5]. The quality of data preparation—including appropriate normalization, batch correction, and removal of low-quality cells—directly impacts annotation accuracy. SingleR's fine-tuning process further refines these annotations by comparing each cell to its nearest neighbors in the reference dataset, requiring properly structured data objects to function effectively [31].

For optimal SingleR performance, ensure that:

• Normalization method is consistent with reference data processing
• Batch effects have been addressed using methods like Harmony or CCA integration [28]
• The data object contains complete gene-level metadata for proper feature alignment with reference datasets

The meticulous preparation of Seurat and SingleCellExperiment objects establishes the critical foundation for successful reference-based cell annotation with SingleR. By following these standardized protocols for quality control, normalization, and data structuring, researchers ensure that their data objects are optimally configured for accurate cell type identification. The interoperability between these object ecosystems further enhances analytical flexibility, enabling researchers to leverage the unique strengths of both Seurat and Bioconductor tools within a unified workflow. As single-cell technologies continue to evolve, with increasing integration of spatial and multi-modal data, these robust data preparation principles will remain essential for extracting biologically meaningful insights from complex cellular systems.

The celldex package is a fundamental resource for reference-based cell type annotation, providing immediate access to a collection of publicly available reference datasets with curated cell type labels. Its primary function is to supply standardized SummarizedExperiment objects for use with automated annotation tools like SingleR [32] [33] [5]. By offering a unified interface to multiple reference datasets, celldex significantly reduces the preliminary data processing burden on researchers, enabling them to focus on the biological interpretation of their single-cell RNA sequencing (scRNA-seq) data. Integrating celldex into a SingleR workflow transforms cell type annotation from a manual, artisanal process into a reproducible, scalable classification procedure, analogous to how genome aligners standardized sequence analysis [5]. This package is essential for researchers, scientists, and drug development professionals who require robust, standardized cellular phenotyping to understand disease mechanisms, identify novel therapeutic targets, and validate cellular models.

The celldex package provides several reference datasets, each meticulously curated and ready for use. The table below summarizes the key characteristics of available primary reference datasets, providing a basis for selection.

Table 1: Core Reference Datasets Available in the celldex Package

Reference Name	Primary Organism	Primary Tissue/Cell Focus	Key Features and Utility
ImmGen [32]	Mouse (10090)	Immune cells	Comprehensive coverage of the murine immune system. Ideal for annotating data from mouse models in immunology and immuno-oncology.
Blueprint/Encode [32]	Human (9606)	Diverse primary cells and tissues	A combined resource from two major projects, providing a broad spectrum of human cell types.
DICE [32]	Human (9606)	Immune cells (PBMCs)	Profiles of human immune cell types under resting and stimulated conditions. Highly relevant for human immunology and biomarker discovery.
HPCA [32]	Human (9606)	Diverse primary cells and tissues	The Human Primary Cell Atlas, another extensive collection of human primary cell types.

These datasets are stored as SummarizedExperiment objects, containing a matrix of log-normalized expression values (logcounts) and critical cell-level metadata. The objects are structured with genes as rows and reference samples as columns. The column metadata includes cell type labels at different resolutions, such as label.main (broad cell type) and label.fine (fine-grained subtype), providing flexibility for annotation specificity [32].

Experimental Protocol: Accessing and Utilizing Reference Datasets

This section provides a detailed, step-by-step protocol for integrating celldex into a SingleR-based cell annotation workflow.

Software Environment Setup

First, ensure the required packages are installed and loaded in your R or Python analysis environment. The celldex package is natively available in R/Bioconductor, with a corresponding Python package (celldex) available via PyPI [32] [33].

For Python Users:

For R Users:

Protocol Steps

Step 1: Discover Available Reference Datasets Begin by listing all references to identify the most current versions and their metadata [32].

Step 2: Search for Relevant References If your study focuses on a specific organism or tissue, use the search function to narrow down options [32].

Step 3: Fetch the Reference Dataset Download your chosen reference dataset. This step retrieves the SummarizedExperiment object into your session [32].

Step 4: Perform Cell Type Annotation with SingleR Use the fetched reference to annotate your single-cell dataset. The following pseudo-code outlines the core logic.

Step 5: Validate Annotation Results Always validate the automated annotation using known marker genes and biological context [31] [34]. UMAP visualization of your query data, colored by the SingleR-predicted labels, can reveal the coherence of the assigned cell types. Cross-reference with the expression of established marker genes for those cell types to confirm the annotations are biologically plausible.

Workflow and Decision Diagrams

The following diagrams illustrate the procedural workflow and logical decision process for using the celldex package effectively.

Diagram 1: Procedural workflow for using the celldex package, from installation to final annotation.

Diagram 2: Decision pathway for selecting the most appropriate reference dataset and annotation resolution based on experimental context.

The Scientist's Toolkit: Essential Research Reagents & Solutions

Table 2: Key Software Tools and Data Resources for Reference-Based Cell Annotation

Tool/Resource	Function in the Workflow	Key Features and Notes
celldex Package [32] [33]	Centralized access to curated reference datasets.	Provides ready-to-use SummarizedExperiment objects, saving weeks of data collection and processing time.
SingleR [31] [5]	Automated cell type annotation algorithm.	Correlates query cell expression profiles with reference data to assign labels. Fast and interpretable.
SummarizedExperiment	Data structure for storing reference and query data.	The standard Bioconductor container for genomic data, ensuring interoperability between packages.
Scanpy/Seurat	Preprocessing and analysis of query scRNA-seq data.	Used for quality control, normalization, and clustering of your data before passing it to SingleR.
Marker Gene Lists [34]	Biological validation of annotation results.	Essential for confirming automated labels. Curate lists from literature or databases for your tissue of interest.

In the workflow of reference-based cell annotation, running the core SingleR() function is a pivotal step where expression profiles from a single-cell experiment are automatically assigned cell type labels. This process transfers biological knowledge from a well-curated reference dataset to a new test dataset, bypassing the need for manual cluster interpretation and marker gene identification [10]. The accuracy of this assignment hinges on a clear understanding of the function's parameters and the underlying computational method. This application note provides a detailed protocol for executing the SingleR() function, interpreting its results, and implementing best practices to ensure biologically meaningful cell type annotations.

The SingleR method can be conceptualized as a robust variant of nearest-neighbor classification. For each cell in the test dataset, it performs the following steps [10]:

• Correlation Calculation: The Spearman correlation between the cell's expression profile and the expression profile of every sample in the reference dataset is computed. This step uses only the union of marker genes identified from pairwise comparisons between labels in the reference, which enhances the resolution between different cell types.
• Scoring: A single score for each reference label is defined as a fixed quantile (default: 0.8) of the correlations across all reference samples with that label. This approach accounts for heterogeneity within reference labels and avoids penalizing labels with many samples.
• Label Assignment: The label with the highest score is chosen as the preliminary prediction for the test cell.
• Fine-Tuning (Optional): An iterative fine-tuning process is performed to improve resolution between closely related labels. The reference is subsetted to include only labels with scores near the maximum, and scores are recomputed using a refined set of marker genes specific to this subset.

Core Parameters of theSingleR()Function

The SingleR() function in R is called with the following fundamental syntax:

The key parameters that control the annotation process are detailed in the table below.

Table 1: Core parameters of the SingleR function and their functions.

Parameter	Data Type	Function	Default Value	Best Practice Guidance
test	Matrix, SummarizedExperiment	The query dataset whose cells need to be annotated.	(Mandatory)	Can be raw (counts) or normalized (log-counts) expression values [14].
ref	Matrix, SummarizedExperiment	The reference dataset with known cell type labels.	(Mandatory)	Should be a high-quality, well-annotated dataset from a similar biological context [35].
labels	Vector	A character vector of cell type labels for each sample (or column) in ref.	(Mandatory)	Can be broad (label.main) or fine-grained (label.fine) from curated references like celldex [14].
quantile	Numeric	The quantile used to compute the per-label score from the correlations.	0.8	Using a high quantile makes the score robust to unrepresentative reference samples [10].
fine.tune	Logical	Controls whether the fine-tuning step is performed.	TRUE	Recommended for distinguishing closely related cell types. Disabling can speed up computation for large datasets [10].
genes	Character	Specifies the gene set used for the initial correlation calculation.	"de" (Differentially Expressed)	The default "de" uses marker genes from the reference, which improves speed and resolution [10].
prune	Logical	Controls whether labels for low-confidence assignments are set to NA.	TRUE	Recommended to automatically filter out ambiguous assignments based on the delta value [9].

Diagnostic Methods for Annotation Quality

After executing SingleR, it is crucial to evaluate the quality of the cell type assignments. The function returns several diagnostics that help assess confidence.

Based on the Scores and Delta within Cells

The primary output includes a matrix of per-cell scores for each reference label (pred$scores) and the assigned labels (pred$labels). A key derived metric is the "delta" (Δ), which is the difference between the score for the assigned label and the median score across all labels for that cell. A low delta indicates an uncertain assignment, possibly because the cell's true type is not in the reference [9].

Table 2: Key diagnostic fields in a SingleR result object.

Diagnostic Field	Description	Interpretation
pred$scores	Matrix of correlation scores for each cell (rows) against each reference label (columns).	Ideally, the assigned label has a score markedly higher than others.
pred$labels	The vector of predicted labels for each cell.	The final cell type assignment.
pred$delta	The difference between the assigned label's score and the median score for that cell.	A higher delta indicates higher confidence in the assignment.
pred$pruned.labels	A version of pred$labels where low-confidence assignments are replaced with NA.	Used to automatically filter out ambiguous cells.

The plotScoreHeatmap() function visualizes the score matrix, allowing for inspection of the spread of scores across cells and labels. Uncertain assignments are seen when a cell has similar scores for a group of labels [9].

Diagram 1: Interpreting Score Heatmaps.

The distribution of delta values across cells can be visualized with plotDeltaDistribution(). Furthermore, the pruning of low-confidence labels can be inspected and customized using the pruneScores() function, which operates on the delta values [9].

Based on Marker Gene Expression

A biologically intuitive diagnostic is to examine the expression of the marker genes that drove the classification in the test dataset. The plotMarkerHeatmap() function visualizes the expression of key marker genes for a specified label [9]. Confidently assigned cells should show strong expression of their label's canonical markers. For example, beta cells in the pancreas should strongly express insulin (INS) [9]. The absence of expected marker expression warrants skepticism about the assignments.

A Practical Workflow for Cell Annotation with SingleR

The following diagram and protocol outline a complete workflow for annotating a single-cell dataset, from data preparation to final diagnosis, using a Seurat object as an example.

Diagram 2: SingleR Annotation Workflow.

Experimental Protocol: Annotating a Seurat Object

• Extract the Expression Matrix: From the Seurat object (seu), extract either raw or normalized counts. With Seurat v5, this is done using the LayerData function.

• Obtain the Reference Data: Acquire a curated reference dataset. The celldex package provides several ready-to-use options.

• Run the Core SingleR Function: Execute the classification with chosen parameters.

• Transfer Labels to the Seurat Object: Add the predictions back to the Seurat object's metadata for downstream analysis and visualization.

• Perform Diagnostic Checks: Generate plots to validate the annotation quality.

The Scientist's Toolkit: Research Reagent Solutions

Table 3: Essential software tools and reference data for SingleR analysis.

Item	Function	Example / Source
SingleR R Package	The core software for performing reference-based cell type annotation.	Available via Bioconductor [5].
Curated Reference Datasets	Pre-annotated bulk and single-cell RNA-seq datasets used as a source of known cell type labels.	celldex package (e.g., HumanPrimaryCellAtlasData, MonacoImmuneData) [14].
SingleCellExperiment Object	A standard S4 class for storing single-cell genomics data, compatible with SingleR.	Used to structure both test and reference data [14].
Seurat Object	A popular alternative class for single-cell data analysis. Requires extraction of an expression matrix for use with SingleR [18] [14].
Diagnostic Plotting Functions	Functions to visualize and validate the annotation results.	plotScoreHeatmap(), plotDeltaDistribution(), plotMarkerHeatmap() [9].

• Reference Dataset Selection: The choice of reference is critical [14]. It should be biologically relevant to the test dataset. Using an inappropriate reference (e.g., a mouse brain reference to annotate human blood cells) will lead to inaccurate assignments. For specialized applications, such as spatial transcriptomics with Xenium data, SingleR has been benchmarked as a top-performing tool when using a matched, high-quality single-nucleus RNA-seq reference [35].
• Handling Large Datasets: For very large test datasets (e.g., >100,000 cells), the fine-tuning step can become computationally expensive. A recommended strategy is to run SingleR on subsets of the data and combine the results [18].
• Integration with Clustering: Comparing the SingleR assignments with the clusters from an unsupervised analysis (e.g., Seurat or SC3 clusters) is highly instructive [9]. Discrepancies can reveal novel cell states or indicate misannotation.

In conclusion, the core SingleR function provides a powerful and efficient method for automated cell type annotation. By understanding its parameters, rigorously performing diagnostic checks, and adhering to best practices—particularly in selecting a high-quality reference—researchers can reliably transfer biological knowledge across datasets, accelerating discovery in genomics and drug development.

In the context of reference-based cell annotation using SingleR, interpreting the results is a critical step that determines the biological validity of the entire analysis. SingleR operates as a robust variant of nearest-neighbors classification, comparing the expression profile of each cell in a query dataset to a reference dataset with known labels [10]. The method calculates Spearman correlations between the test cell and all reference samples, defines a per-label score as a fixed quantile (default: 0.8) of these correlations, and optionally performs fine-tuning to improve resolution between closely related labels [10]. This protocol focuses on the crucial diagnostic measures—scores, labels, and delta values—that researchers must understand to validate their cell type assignments confidently. Proper interpretation of these metrics ensures that subsequent biological conclusions are built upon a reliable cellular foundation.

Understanding the Scoring System

The Scores Matrix

The foundational diagnostic reported by SingleR() is the nested matrix of per-cell scores in the scores field. This matrix contains the correlation-based scores for each cell (row) against every reference label (column) prior to any fine-tuning [9]. The following table summarizes the key characteristics and interpretation guidelines for the scores matrix:

Table 1: Interpretation of SingleR Scores Matrix

Aspect	Description	Interpretation Guideline
Origin	Pre-tuning correlation scores	Scores after fine-tuning are not directly comparable across all labels
Ideal Pattern	One label's score is clearly larger than others for each cell	Indicates unambiguous assignment
Problematic Pattern	Similar scores for multiple labels	Suggests uncertain assignment or closely related cell types
Visualization	plotScoreHeatmap() function	Adjusts values to highlight differences within cells

Examination of this matrix should focus on the spread of scores within each cell. Ideally, for any given cell, one label's score should be substantially larger than the others, indicating a confident assignment [9]. For example, an initial examination might show:

In this example, the first cell shows the highest score for acinar (0.7312), but duct cells also show a moderately high score (0.5527), suggesting some potential ambiguity that warrants further investigation.

Visualizing Scores with Heatmaps

The plotScoreHeatmap() function provides an effective visualization of the score matrix, where each column represents a cell and each row represents a reference label [9]. This heatmap does not faithfully represent the absolute score values but instead adjusts them to highlight differences between labels within each cell, making it easier to spot ambiguous assignments.

Diagram 1: Workflow for Score Heatmap Interpretation

The heatmap can be enhanced by setting clusters= or annotation_col= parameters to display additional metadata, such as donor of origin or unsupervised clustering results, which helps identify potential batch effects or validate against independent groupings [9].

Confidence Assessment Using Delta Values

The Delta Metric

The delta value represents a crucial confidence metric in SingleR, defined as the difference between the score for the assigned label and the median across all labels for each cell [9] [36]. This metric operates on the assumption that most reference labels are not relevant to any given cell, making the median a useful baseline correlation measure. The gap between the assigned label's score and this baseline indicates assignment confidence.

The mathematical calculation is straightforward: Δ = Scoreassignedlabel - Median(Scorealllabels)

Table 2: Delta Value Interpretation and Thresholding

Delta Value	Interpretation	Recommended Action
High Δ	Confident, unambiguous assignment	Include in downstream analysis
Low Δ	Uncertain assignment; true cell type may not be in reference	Consider pruning or flagging
Very Low Δ	Poor-quality assignment or unknown cell type	Prune from final annotation

Low delta values indicate that a cell matches all labels with similar confidence, suggesting the assigned label has low significance [36]. This commonly occurs when a cell's true type is absent from the reference dataset.

Pruning Strategies

SingleR implements an automated pruning approach that identifies cells with delta values that are small outliers relative to other cells with the same label [9]. This method assumes that for any given label, most assigned cells are correct. The results are reported in the pruned.labels field, where low-quality assignments are replaced with NA.

The default outlier-based pruning may not be appropriate for all datasets, particularly when one label is consistently misassigned. In such cases, a fixed threshold can be applied using the pruneScores() function with the min.diff.med= parameter [9]:

Visualizing Delta Distributions

The plotDeltaDistribution() function generates a visualization showing the per-label distribution of delta values across cells, allowing researchers to verify that outlier detection in pruneScores() behaved appropriately [9]. Labels with consistently low delta values warrant additional caution in biological interpretation.

Diagram 2: Delta Analysis and Pruning Workflow

Validation with Marker Gene Expression

Marker Gene Heatmaps

A biologically intuitive diagnostic involves examining the expression of marker genes used for annotation in the test dataset. The plotMarkerHeatmap() function automatically visualizes expression of the most relevant markers for a specified label—those upregulated in the test dataset and responsible for driving classification to that label [9].

For example, when examining beta cell assignments:

A confident assignment to beta cells should show strong expression of canonical markers like insulin (INS) in the assigned cells [9]. If identified markers are not meaningful or not consistently upregulated, this warrants skepticism about assignment quality.

Systematic Marker Validation

For comprehensive validation, researchers can create diagnostic plots for each label by iterating through all cell types:

This approach facilitates quick assessment of assignment quality across all annotated cell types. The heatmap configuration from configureMarkerHeatmap() can be reused with other plotting functions like plotDots() from scater or dittoHeatmap() from dittoSeq for customized visualizations [9].

Integration with Unsupervised Clustering

Comparing SingleR assignments to unsupervised clustering provides an independent validation of the annotation quality. The assumption is that biologically distinct cell types should form separate clusters in unsupervised analysis [9]. Discrepancies between SingleR assignments and unsupervised clusters may indicate:

• Over-clustering: SingleR distinguishes subtypes not separated in unsupervised clustering
• Under-clustering: Unsupervised methods separate cells assigned to the same label
• Annotation errors: SingleR may be incorrectly assigning distinct cell types

This comparison can be visualized by setting the clusters= parameter in plotScoreHeatmap() to display unsupervised clustering results alongside SingleR scores [9].

Research Reagent Solutions

Table 3: Essential Tools for SingleR Annotation Validation

Tool/Resource	Function	Application Context
SingleR package [9]	Primary annotation algorithm	Reference-based cell type assignment
celldex [10]	Reference dataset collection	Provides curated reference data (Blueprint/ENCODE, etc.)
plotScoreHeatmap() [9]	Score visualization	Identifying ambiguous assignments
plotDeltaDistribution() [9]	Delta value assessment	Evaluating assignment confidence
plotMarkerHeatmap() [9]	Marker expression validation	Biological verification of assignments
Human Primary Cell Atlas [10]	Reference dataset	Immune and common cell type annotation
Blueprint/ENCODE [37]	Reference dataset	Human tissue cell type annotation
pruneScores() [9]	Quality filtering	Removing low-confidence assignments

Benchmarking and Performance Context

In benchmarking studies comparing reference-based annotation methods for spatial transcriptomics data, SingleR emerged as the best-performing tool, being "fast, accurate and easy to use, with results closely matching those of manual annotation" [35]. This independent validation underscores the reliability of SingleR's scoring system when properly interpreted.

The interpretation framework outlined in this protocol enables researchers to leverage SingleR's performance advantages while maintaining critical assessment of the resulting annotations, ensuring biological validity in downstream analyses.

Interpretation of SingleR Output Data Structure

The SingleR() function returns a complex object whose correct interpretation is critical for downstream analysis. The table below summarizes the key output fields and their biological and computational significance.

Table 1: Key Output Fields from the SingleR Function

Output Field	Data Class	Description	Primary Downstream Use
labels	character	The primary predicted cell type label for each cell.	Core metadata for coloring UMAP/t-SNE plots and defining cluster identities.
scores	matrix	A matrix of correlation scores for each cell (rows) against every reference label (columns) prior to fine-tuning.	Diagnosing assignment confidence and ambiguity between related cell types.
pruned.labels	character	A version of labels where low-confidence assignments are replaced with NA.	Filtering out noisy cells before differential expression or trajectory analysis.
delta.next	numeric	The difference between the score for the assigned label and the score for the next-best label.	A direct metric of confidence for distinguishing between the two most similar cell types.

The scores matrix is particularly valuable for diagnostics. Each row represents a single cell, and each column a reference label. Ideally, the assigned label for a cell should have a score significantly higher than all other labels in its row. The pruned.labels field uses an outlier-based detection method to automatically identify and remove low-confidence assignments, replacing them with NA [9].

Quality Control and Diagnostic Checks for Predictions

Rigorous quality control is essential before accepting SingleR's labels. The following diagnostic checks should be performed to assess confidence and biological plausibility.

Visualizing the Score Distribution

The plotScoreHeatmap() function visualizes the matrix of pre-tuned scores, highlighting the spread of scores for each cell. This allows for the identification of cells with ambiguous assignments, where multiple labels have similar correlation scores. This uncertainty may be acceptable if it occurs between biologically related cell types (e.g., T cell subtypes) but warrants investigation if it occurs between distinct lineages [9].

Assessing the Per-Cell "Delta"

A more robust measure of confidence is the "delta", defined as the difference between the score for the assigned label and the median score across all labels for that cell. A low delta indicates an uncertain assignment, potentially because the cell's true type is absent from the reference. The plotDeltaDistribution() function visualizes the distribution of these deltas for each assigned label, helping to identify cell types with systematically low confidence. Cells with delta values below a defined threshold (e.g., via pruneScores(min.diff.med=0.2)) should be considered for removal [9].

Validation with Marker Gene Expression

Correlation with reference data is informative, but biological validation is paramount. The plotMarkerHeatmap() function extracts the key marker genes that SingleR used for classification and visualizes their expression in the test dataset. A successful and biologically meaningful annotation will show strong, specific expression of canonical marker genes (e.g., insulin (INS) in beta cells) in the clusters assigned to the corresponding label [9].

Diagram 1: SingleR Prediction Quality Control Workflow

Protocol: Integration into a SingleCellExperiment Object

This protocol details the steps for integrating SingleR predictions and their associated diagnostics into a SingleCellExperiment (SCE) object for seamless downstream analysis.

Materials:

• SingleR Output Object: The result from the SingleR() function.
• SingleCellExperiment Object: The SCE object containing the single-cell data used for the SingleR prediction.
• R Environment: R (version 4.5.1 or higher) with the following packages installed: SingleR, SingleCellExperiment, Matrix.

Procedure:

• Transfer Primary Labels: Add the primary predicted labels to the colData of your SingleCellExperiment object. This makes the cell types available for coloring plots and defining groups.

• Incorporate Pruned Labels: Add the pruned labels to safely exclude low-confidence assignments in specific analyses.

• Add Confidence Metrics: Store the delta values for each cell to enable filtering or coloring by confidence.

• (Optional) Store Full Scores Matrix: For advanced diagnostics, the entire scores matrix can be stored in the SCE's metadata.

Table 2: Research Reagent Solutions for SingleR Integration

Item	Function in Protocol
SingleCellExperiment Object	The primary container for the single-cell dataset, holding expression data, cell metadata, and gene annotations.
SingleR Output Object	The object returned by the SingleR() function, containing all prediction results and diagnostics.
$ and [[ Operators	R operators used to access and assign new columns within the colData of the SCE object.
metadata() Function	An accessor/getter function used to store and retrieve the full, complex scores matrix within the SCE object for later diagnostics.

Visualization and Downstream Analysis

With cell types integrated, researchers can proceed to biologically insightful visualizations and analyses.

Dimensionality Reduction Plots

Color a UMAP or t-SNE plot using the SingleR.labels column. This provides an immediate visual assessment of the relationship between clustering and automated cell type annotation. Cells can also be colored by SingleR.delta to visually identify clusters or regions with low-confidence annotations.

Annotated Cluster Analysis

Compare the SingleR labels with unsupervised clustering results. This validates the annotation against an independent method and can reveal potential substructure within an annotated cell population.

Differential Expression and Marker Discovery

Use the validated cell type labels as groups for differential expression analysis. This identifies genes that are significantly upregulated in each cell type within your specific dataset, complementing the reference-based annotation with data-driven discovery.

Diagram 2: Downstream Visualization and Analysis

Solving Common SingleR Challenges: Speed, Accuracy, and Reference Mismatch

Reference-based cell annotation with SingleR is a powerful method for assigning cell type identities to single-cell RNA sequencing (scRNA-seq) data. As dataset sizes grow exponentially, computational efficiency becomes crucial for practical analysis. This protocol explores two primary acceleration strategies: parallelization using the BiocParallel framework to distribute computations across multiple processors, and cluster-level annotation to reduce computational burden by aggregating cells before classification. These approaches maintain SingleR's renowned accuracy while significantly decreasing processing time, enabling researchers to analyze large-scale datasets efficiently. SingleR operates by comparing gene expression profiles of single cells to reference datasets with predefined labels, using correlation-based methods to identify the most likely cell type for each cell [38] [11]. The framework's flexibility allows for optimization at various stages of the computational pipeline, which we will explore in detail throughout this application note.

Parallelization with BiocParallel

Implementation Framework

The BiocParallel package provides a standardized interface for parallel execution across various computing environments. SingleR seamlessly integrates with this framework through the BPPARAM parameter, enabling researchers to distribute the computational workload of cell type annotation without modifying their core analysis code [39]. This implementation is particularly valuable for large datasets where processing individual cells sequentially would be prohibitively time-consuming. The parallelization approach distributes cells across available processing cores, with each core independently calculating correlation scores against the reference dataset, thereby reducing overall computation time proportional to the number of cores utilized.

To implement parallel processing with SingleR, researchers can select from several parallel backends:

• MulticoreParam: Utilizes process forking for parallel execution on POSIX-compliant systems (Linux and MacOS). This backend offers minimal overhead but is unavailable on Windows systems [39].
• SnowParam: Implements parallelization using separate processes, compatible with all operating systems including Windows. While slightly slower than forking due to inter-process communication overhead, it provides broader system compatibility [39].
• BatchtoolsParam: Interfaces with cluster job schedulers like SLURM, LSF, and others, enabling large-scale parallelization across high-performance computing environments. This backend requires configuration specific to the computing environment but supports distribution across hundreds of CPUs for extremely demanding jobs [39].

Practical Implementation

The implementation of parallel processing requires minimal code modification. Researchers simply specify their chosen parallel backend via the BPPARAM parameter in their SingleR function call:

Table 1: Comparison of BiocParallel Parallelization Backends

Parameter Type	Compatible Systems	Key Features	Optimal Use Cases
MulticoreParam	Linux, MacOS	Minimal overhead through forking	Single-machine processing of large datasets
SnowParam	All systems (including Windows)	Socket-based parallelization	Cross-platform analyses; smaller datasets
BatchtoolsParam	HPC clusters with job schedulers	Integration with SLURM, LSF, etc.	Extremely large datasets (>1 million cells)

The effectiveness of parallelization depends on several factors, including dataset size, reference complexity, and available computational resources. Benchmarking tests demonstrate that parallelization can reduce computation time by approximately 60-80% when utilizing 8 cores compared to sequential processing, with diminishing returns observed beyond 16 cores for most dataset sizes [39]. For very large datasets exceeding 100,000 cells, the performance gains can be even more substantial, potentially reducing processing time from hours to minutes.

Cluster-Level Annotation

Conceptual Framework and Workflow

Cluster-level annotation represents an alternative acceleration strategy that reduces computational burden by aggregating cells into clusters before annotation. Rather than classifying individual cells, SingleR calculates an aggregated expression profile for each cluster and assigns a single cell type label to the entire group [39]. This approach significantly decreases the number of comparisons required, as a dataset with 10,000 cells clustered into 30 groups would require 30 classification operations instead of 10,000.

The underlying assumption of this method is that cells within a cluster share the same cell type identity, which generally holds true for well-separated clusters but may break down in cases of continuous differentiation or poorly separated cell types. This approach is particularly valuable during initial exploratory analysis or when working with extremely large datasets where computational resources are constrained.

The following diagram illustrates the workflow for cluster-level annotation:

Implementation Protocol

Implementation of cluster-level annotation requires pre-existing cluster assignments, which can be generated using any standard scRNA-seq clustering method such as those available in Seurat or Scran. The following code demonstrates the practical implementation:

The clusters parameter directs SingleR to compute a single aggregated profile per cluster using the mean normalized expression values across all cells within that cluster. Annotation then proceeds using these cluster-level profiles rather than individual cell profiles [39]. The output provides one annotation per cluster, which can be propagated to all cells within each cluster.

Table 2: Comparison of Individual Cell vs. Cluster-Level Annotation

Characteristic	Per-Cell Annotation	Cluster-Level Annotation
Computational demand	High (scales with cell number)	Low (scales with cluster number)
Resolution	Single-cell level	Cluster level
Handling of mixed populations	Identifies subtle differences	May miss heterogeneity
Interpretation	Can be ambiguous for intermediate cells	Clear cluster identity
Optimal use cases	Identifying rare populations, continuous processes	Initial analysis, large datasets, well-separated populations

Cluster-level annotation typically achieves a 20-50x speed improvement compared to per-cell annotation for typical datasets, with the exact improvement factor dependent on the average cluster size [39]. This approach excels when clusters correspond to distinct cell types but may oversimplify biological complexity in cases of continuous differentiation or when multiple cell types are contained within a single cluster.

Integrated Acceleration Protocol

Combined Workflow

For maximum efficiency, researchers can implement both parallelization and cluster-level annotation simultaneously. This combined approach leverages the reduced computational workload of cluster-level analysis with the distributed processing capabilities of parallelization. The following workflow provides a comprehensive protocol for accelerated cell type annotation:

Step-by-Step Protocol

• Data Preprocessing: Perform standard scRNA-seq quality control and normalization using preferred methods (e.g., scuttle, scran). Remove low-quality cells and genes before proceeding to clustering.

• Cell Clustering: Generate cluster assignments using a computationally efficient method. The quickCluster function from scran provides rapid clustering suitable for this purpose:

• Parallel Configuration: Select an appropriate BiocParallel parameter based on your computing environment:

• Cluster-Level Annotation: Execute SingleR with both cluster assignments and parallel parameters:

• Result Propagation: Apply cluster-level annotations to individual cells for downstream analysis:

Validation and Quality Control

After implementing accelerated annotation, validation is essential to ensure biological accuracy:

• Compare with full analysis: Run standard per-cell annotation on a subset of data to verify cluster-level results.
• Examine marker expression: Validate assigned labels by visualizing expression of known marker genes.
• Assess confidence scores: Examine SingleR's built-in confidence scores for each cluster annotation.

Cluster-level annotation performs best when reference labels are well-separated and cluster definitions align with biological cell types [39]. Performance may decrease when distinguishing closely related cell types or when clustering does not reflect true biological populations.

Performance Benchmarks

Quantitative Assessment

Rigorous benchmarking demonstrates the performance improvements achievable through these acceleration strategies. The following table summarizes typical performance gains across different dataset sizes:

Table 3: Performance Benchmarks for SingleR Acceleration Methods

Dataset Size	Standard SingleR	Parallel Only (8 cores)	Cluster-Level Only	Combined Approach
5,000 cells	1x (reference)	2.8x faster	35x faster	42x faster
20,000 cells	1x (reference)	3.1x faster	42x faster	48x faster
100,000 cells	1x (reference)	3.5x faster	47x faster	52x faster

These benchmarks were conducted on a Linux system with 16 CPU cores and 64GB RAM, using a reference dataset with 25 main cell types [39]. The combined approach demonstrates synergistic effects, with parallelization effectively reducing the overhead of cluster profile calculation and comparison.

Accuracy Considerations

While acceleration methods significantly improve computational efficiency, researchers must consider potential impacts on annotation accuracy:

• Cluster-level annotation assumes homogeneous cell type composition within clusters, which may not reflect biological reality in cases of:

  • Continuous differentiation trajectories
  • Poorly separated cell types
  • Rare cell populations that don't form distinct clusters
  • Mixed populations due to clustering artifacts

• Parallelization maintains identical results to sequential processing, as it merely distributes the same computations across multiple cores.

Studies demonstrate that cluster-level annotation maintains >95% concordance with per-cell annotations for well-separated cell types in PBMC datasets [39]. However, accuracy decreases to 70-80% for closely related T-cell subsets or continuous differentiation processes, highlighting the importance of method selection based on biological context.

The Scientist's Toolkit

Essential Research Reagent Solutions

Table 4: Essential Computational Tools for Accelerated SingleR Analysis

Tool/Resource	Function	Application Context
BiocParallel	Parallel execution framework	Distributing computations across cores
scran	Efficient scRNA-seq analysis	Rapid cell clustering for cluster-level annotation
DICE/Blueprint/ENCODE	Reference datasets	Well-curated reference for immune cell annotation
Human Cell Atlas	Comprehensive reference atlas	Tissue-specific cell type annotation
Seurat	scRNA-seq analysis toolkit	Alternative clustering and visualization
- SingleR Package	Core annotation algorithm	Reference-based cell type identification

These tools collectively provide a comprehensive ecosystem for efficient cell type annotation. The BiocParallel framework serves as the foundation for parallelization, while clustering tools like scran enable rapid cluster definition for cluster-level analysis [39]. Reference datasets such as DICE provide well-curated expression profiles for accurate annotation, particularly for immune cells [39].

The acceleration strategies presented in this protocol—parallelization with BiocParallel and cluster-level annotation—significantly enhance the scalability of SingleR for large-scale scRNA-seq studies. By strategically implementing these approaches, researchers can reduce computation time from hours to minutes while maintaining biologically meaningful results. The combined approach is particularly powerful for large dataset exploration, screening analyses, and situations with limited computational resources. As single-cell technologies continue to evolve toward increasingly large datasets, these optimization strategies will become increasingly essential in the researcher's toolkit, enabling comprehensive analysis while managing computational costs.

Within the framework of research on reference-based cell annotation using SingleR, managing computational resources is a critical challenge. As single-cell RNA sequencing (scRNA-seq) datasets grow to encompass millions of cells, traditional analysis pipelines face significant bottlenecks in computation time and memory usage [40]. This document details practical protocols for leveraging GPU acceleration and approximate nearest neighbor (ANN) algorithms to dramatically enhance the efficiency and scalability of the SingleR annotation workflow without substantially compromising accuracy. These strategies are essential for conducting large-scale studies, such as the construction of cellular atlases or the integrative analysis of multiple datasets.

The following tables summarize key performance metrics and resource requirements for the technologies discussed in this protocol.

Table 1: Benchmarking Data for Computational Tools in Single-Cell Analysis

Tool / Algorithm	Dataset Size	Hardware Configuration	Time	Comparative Performance
ScaleSC [40]	1.3 million cells	1 TB CPU RAM, 1x NVIDIA A100 GPU	2 minutes	135x faster than Scanpy
Scanpy [40]	1.3 million cells	CPU-based	4.5 hours	Baseline (1x)
ScaleSC [40]	13 million cells	1 TB CPU RAM, 1x NVIDIA A100 GPU	~1 hour	Surpasses rapids-singlecell
SingleR [35]	10x Xenium data (Imaging-based)	Information Not Specified	Fast	Best performing tool in benchmark

Table 2: Key Specifications of Selected NVIDIA GPUs for Single-Cell Analysis

GPU Model	Architecture	VRAM	Memory Bandwidth	Key Feature for Single-Cell Analysis
A100 [41]	Ampere	80 GB HBM2	1,555 GB/s	High memory for large datasets; Tensor Cores for accelerated matrix math.
H100 [42]	Hopper	80 GB HBM3	3.35 TB/s	FP8 precision support, ~2 PFLOPS for AI workloads.
RTX 4090 [41]	Ada Lovelace	24 GB GDDR6X	1 TB/s	Cost-effective for medium-scale models; high CUDA core count.

Table 3: Comparison of Approximate Nearest Neighbor (ANN) Algorithms

Algorithm	Type	Primary Data Structure	Key Characteristic
HNSW [43]	Graph-based	Proximity Graph	Fast and high-recall; used in modern vector databases.
Annoy [44]	Tree-based	Binary Search Tree Forest	Focuses on high recall; provides static, read-only indexes.
FAISS [43]	Multiple	Various (e.g., IVF, PQ)	Highly optimized library from Meta; supports CPU and GPU.
Product Quantization (PQ) [43]	Compression-based	Compressed Vectors	Reduces memory footprint by compressing high-dimensional vectors.

Experimental Protocols

Protocol 1: Accelerating SingleR Annotation with GPU-Accelerated Pipelines

This protocol describes integrating SingleR with a GPU-accelerated preprocessing pipeline like ScaleSC to reduce computation time from hours to minutes.

Methodology:

• Data Input and Chunking: Load the single-cell gene expression matrix (query dataset) using a chunking strategy. ScaleSC overcomes memory bottlenecks by dividing the dataset into manageable chunks that are processed incrementally, rather than loading the entire dataset into memory at once [40].
• GPU-Accelerated Preprocessing: On a system with a compatible GPU (e.g., NVIDIA A100), perform key preprocessing steps. ScaleSC implements new GPU-powered algorithms for:
  • Cell Quality Control (QC): Filtering cells based on metrics like mitochondrial gene percentage.
  • Highly Variable Gene (HVG) Detection: Identifying genes that show high cell-to-cell variation.
  • Principal Component Analysis (PCA): Linear dimensionality reduction [40].
• Reference Dataset Processing: Independently, preprocess the labeled reference dataset (e.g., from cellxgene) using the same GPU-accelerated pipeline to ensure consistency. It is critical that the preprocessing steps and gene space between the query and reference datasets are aligned [45].
• Cell Type Annotation with SingleR: Transfer the preprocessed, chunked query data and the preprocessed reference data to the SingleR tool. SingleR will perform a correlation-based analysis between cells in the query dataset and the reference dataset to assign cell type labels [35]. The input to SingleR is now optimized for speed, while the core annotation algorithm remains the same.

Key Hardware Considerations:

• System Requirements: The benchmark for ScaleSC used a system with 1 TB of CPU RAM and a single NVIDIA A100 GPU [40]. For large datasets (>1 million cells), a GPU with high VRAM (>40 GB) is recommended.
• GPU Selection: The choice of GPU can be guided by Table 2. For instance, the A100's high VRAM is suited for massive datasets, while the H100 offers superior memory bandwidth for faster data processing [42] [41].

Protocol 2: Integrating Approximate Nearest Neighbor Search with SingleR

This protocol modifies the SingleR workflow by replacing the default exact nearest neighbor search with an ANN algorithm like Annoy to reduce computational load during the correlation step.

Methodology:

• Standard Preprocessing: Normalize and log-transform both the query and reference datasets using a standard CPU-based pipeline (e.g., with Scanpy in Python) [40].
• Building the ANN Index: Instead of using the full reference dataset directly, build an ANN index from the reference data.
  • Algorithm Selection: Choose an ANN algorithm such as Annoy (Approximate Nearest Neighbors Oh Yeah) [44].
  • Index Construction with Annoy: Annoy operates by constructing a forest of binary trees. It recursively partitions the vector space by generating hyperplanes equidistant from two randomly chosen data points, assigning vectors to the left or right subtree based on proximity. This process continues until leaf nodes contain a predefined maximum number of vectors (K) [44]. The following code illustrates the conceptual process:

• Querying for Nearest Neighbors: For each cell in the query dataset, use the ANN index to find its approximate nearest neighbors in the reference dataset.
  • Query Execution: The query traverses each tree in the forest to find the leaf node closest to the query vector. The final candidate set is the union of all vectors from the identified leaf nodes across all trees, which is then used to compute the final nearest neighbors [44].
• Cell Type Label Transfer: Use the identities of the approximate nearest neighbors found in the reference dataset to assign a cell type label to each query cell, following the standard SingleR methodology [35].

Workflow Visualization

The following diagrams illustrate the core workflows described in the experimental protocols.

GPU-Accelerated SingleR Analysis

ANN-Enhanced SingleR Annotation

The Scientist's Toolkit

Table 4: Essential Research Reagents and Computational Tools

Item Name	Function / Application in Workflow
ScaleSC [40]	A GPU-accelerated scRNA-seq data analysis pipeline used for superfast preprocessing (QC, HVG, PCA) before annotation.
SingleR [35]	A reference-based cell type annotation tool that correlates query cells with a reference dataset to assign cell type labels.
Annoy (ANN Algorithm) [44]	A library for approximate nearest neighbor search using a forest of binary trees; used to speed up the neighbor-finding step in SingleR.
Scanpy [40]	A standard Python-based toolkit for analyzing single-cell gene expression data, often used for CPU-based preprocessing and analysis.
NVIDIA A100 GPU [40] [41]	A high-performance GPU with large VRAM (80GB), providing the computational power for accelerating large-scale single-cell analysis.
Reference Dataset (e.g., from cellxgene) [45]	A well-annotated single-cell dataset used as a ground truth for transferring cell type labels to a new, unannotated query dataset.

In reference-based cell type annotation with SingleR, simply obtaining cell labels is only the first step. A critical, often overlooked, phase is the diagnostic assessment of these assignments to identify and handle low-confidence predictions. SingleR automatically evaluates the confidence of each cell-to-label assignment, flagging ambiguous or low-quality annotations that could otherwise introduce noise into downstream analyses [9]. This protocol focuses on two core diagnostic concepts: the delta score, a measure of assignment confidence, and pruned labels, which are the result of automatically filtering out low-confidence assignments. Mastering the interpretation and handling of these metrics is essential for generating robust, reproducible cell type annotations in single-cell RNA sequencing (scRNA-seq) studies relevant to drug development and disease research.

Understanding Key Diagnostic Metrics

The Delta Score

The delta score for a cell is defined as the difference between the score for its assigned label and the median score across all possible reference labels for that cell [9]. This metric operates on the principle that the majority of reference labels are not relevant to any given cell. The median score thus represents a baseline level of correlation, and the delta quantifies how far the best assignment rises above this baseline.

• Interpretation: A high delta score indicates an unambiguous assignment, where the cell's expression profile strongly and specifically matches its assigned label. A low delta score suggests an uncertain assignment, which can occur if the cell's true type is absent from the reference, the cell is of low quality, or the cell represents an intermediate or hybrid state [9].
• Robustness: The delta is preferred over the raw assignment score because it is more robust to technical effects. Changes in library size or other technical factors can raise or lower all scores for a cell uniformly, but the delta, being a within-cell difference, remains relatively stable [9].

Pruned Labels

Pruned labels are the outcome of applying an automated filter to remove low-confidence assignments. In the SingleR output, these are reported in the pruned.labels field, where low-quality assignments are replaced with NA [9].

SingleR's default pruning method uses an outlier-based strategy for each label independently. It identifies cells with deltas that are small outliers compared to the deltas of other cells assigned to the same label. This strategy relies on the assumption that, for a given label, the majority of assigned cells are correct. The default parameters may not be suitable for all datasets, particularly if an entire label is consistently misassigned [9].

Protocols for Diagnostic Analysis

Protocol 1: Assessing Score Distributions and Default Pruning

This protocol provides a baseline assessment of annotation confidence.

• Run SingleR: Perform cell type annotation using the SingleR() function with your test dataset and a chosen reference.
• Access Diagnostics: The pruned.labels are found in the pruned.labels field of the returned SingleR object. The per-cell scores and deltas are in the scores and delta fields, respectively [9].
• Visualize Distributions: Use plotDeltaDistribution(pred.grun) to generate a plot showing the distribution of delta scores for all cells assigned to each label. This allows for a quick visual assessment of which labels have consistently low deltas [9].
• Summarize Pruning: Create a table summarizing the number of cells for which labels were kept or removed for each cell type.

Expected Output: The table below summarizes the results of applying default pruning to a pancreas dataset [9]:

Table 1: Example Summary of Default Pruning on a Pancreas Dataset

Label	Cells Retained	Cells Pruned
acinar	260	29
alpha	200	1
beta	177	1
delta	52	2
duct	291	4
endothelial	5	0
epsilon	1	0
mesenchymal	22	1
pp	18	0

Protocol 2: Manual Thresholding on Delta Scores

For cases where default pruning is inadequate, implement a fixed threshold.

• Obtain Delta Values: Extract the delta score for each cell from the SingleR result.
• Set a Threshold: Manually call the pruneScores() function, setting the min.diff.med argument to your chosen delta threshold (e.g., min.diff.med = 0.2). Higher thresholds enforce greater certainty [9].
• Apply and Evaluate: Apply the pruning and generate a new summary table. Compare the results to the default pruning to understand the impact of a more or less stringent threshold.

Expected Output: The table below shows how a fixed delta threshold of 0.2 affects pruning compared to the default method [9]:

Table 2: Pruning Comparison: Default vs. Fixed Delta Threshold (0.2)

Label	Default (Retained)	Fixed Threshold (Retained)
acinar	260	259
alpha	200	168
beta	177	149
delta	52	37
duct	291	291
endothelial	5	5
epsilon	1	1
mesenchymal	22	22
pp	18	5

Protocol 3: Fine-Tuning Based Filtering

After fine-tuning, a more stringent filter can be applied based on the difference between the highest and next-highest scores.

• Check for Fine-Tuning: Ensure SingleR was run with the fine-tuning step enabled (it is by default).
• Apply Fine-Tune Filter: Use pruneScores() with the tune.thresh argument set to TRUE. This will prune assignments where the winning label is not clearly distinguishable from the second-best label [9].
• Interpret Results: Be aware that this method is often conservative and can heavily penalize assignments involving closely related cell types (e.g., T cell subsets, neuronal subtypes) [9].

Diagram 1: SingleR Diagnostic Workflow

Visual and Biological Diagnostics

Visualizing Scores and Deltas

• Heatmap of Scores: The plotScoreHeatmap() function visualizes the matrix of per-cell scores for each label. The key is to examine the spread of scores within each cell (columns). Similar scores for a group of labels indicate uncertain assignment for those cells, though this may be acceptable if the uncertainty is among related types [9].
• Delta Distribution: As detailed in Protocol 1, plotDeltaDistribution() is the primary tool for visualizing the per-label spread of deltas, allowing for easy identification of labels with generally low confidence [9].

Marker Gene Expression Validation

A biologically intuitive diagnostic is to check the expression of the marker genes that drove the classification in the test dataset.

• Retrieve Markers: The marker genes used for each label are stored in the metadata() of the SingleR result [9].
• Visualize Expression: Use plotMarkerHeatmap(pred.grun, sceG, "beta") to create a heatmap showing the expression of the most relevant markers for a specified label (e.g., "beta" cells) in the test data [9].
• Biological Validation: Confidently assigned cells should show strong expression of their label's canonical markers. For example, beta cells should show high expression of insulin (INS). If the identified markers are not meaningful or not consistently upregulated, the assignments should be treated with skepticism [9].

Diagram 2: Causes and Actions for Low Delta

The Scientist's Toolkit

Table 3: Essential Research Reagent Solutions for SingleR Annotation

Item	Function
Reference Datasets (e.g., Human Primary Cell Atlas, ImmGen)	Provides the curated, pre-annotated expression profiles against which test cells are compared. The choice of reference is critical for annotation accuracy [10] [31].
Test scRNA-seq Dataset	The query dataset containing unlabeled cells, typically formatted as a SingleCellExperiment or Seurat object.
SingleR Software Package	The primary tool for performing reference-based annotation and calculating per-cell scores and delta metrics [9] [10].
Visualization Packages (e.g., scater, dittoSeq)	Used to create diagnostic plots such as plotScoreHeatmap(), plotDeltaDistribution(), and plotMarkerHeatmap() to interpret and validate results [9].
Marker Gene Lists	Curated sets of canonical genes for expected cell types; used for biological validation of SingleR assignments and for interpreting clustering [9] [46].

Strategies for Reference Dataset Selection and Handling Missing Cell Types

SingleR is an automated computational method for cell type recognition in single-cell RNA sequencing (scRNA-seq) data that leverages reference transcriptomic datasets of pure cell types to infer the cell of origin of each single cell independently [18]. As a robust variant of nearest-neighbors classification, SingleR operates by comparing the gene expression profile of each test cell to reference samples with known labels, assigning labels based on the highest similarity in expression patterns [10]. The method transfers biological knowledge across datasets, allowing researchers to propagate expertly curated annotations from reference datasets to new experimental data in a systematic, automated manner [10]. This approach significantly reduces the burden of manually interpreting clusters and defining marker genes for each new dataset.

The performance of SingleR and similar reference-based annotation tools depends critically on two factors: the selection of an appropriate reference dataset and strategies for handling cell types that may be absent from the reference. This protocol provides comprehensive guidance for researchers navigating these critical decisions, with particular emphasis on practical implementation within the context of a broader thesis on reference-based cell annotation. We include detailed methodologies for reference evaluation, experimental protocols for validation, and visualization tools to support researchers in making informed decisions throughout the annotation workflow.

Theoretical Foundations of Reference Dataset Selection

How SingleR Utilizes Reference Data

SingleR's algorithm functions through a multi-step process designed to maximize annotation accuracy while maintaining computational efficiency. For each test cell, the method first computes the Spearman correlation between its expression profile and each reference sample, using only the union of marker genes identified by pairwise comparisons between labels in the reference data [10]. This focused approach improves resolution for separating closely related labels. The algorithm then defines a per-label score as a fixed quantile (default: 0.8) of the correlations across all samples with that label, which accounts for differences in reference sample numbers and avoids penalizing classifications to heterogeneous labels [10].

An optional fine-tuning step iteratively improves resolution between closely related labels by subsetting the reference to only include labels with scores near the maximum and recomputing scores using marker genes specific to the subset of labels [10]. This process continues until only one label remains. The method can operate in either "classic" mode, which uses log-fold changes for marker detection (primarily for bulk-derived references with limited replication), or in single-cell mode, which employs conventional statistical tests like Wilcoxon rank sum tests to account for cellular variability [47] [8].

Impact of Reference Quality on Annotation Performance

The choice of reference dataset fundamentally impacts annotation results, as SingleR requires a reference that contains a superset of the labels expected in the test dataset [8]. References with inappropriate or low-quality labels can propagate errors through the annotation process, while well-curated references enable accurate transfer of biological knowledge. The key limitation of this approach emerges when test datasets contain cell types completely absent from the reference, which can lead to misannotation or the problematic grouping of distinct cell types under similar labels [31].

Studies comparing annotation methods have demonstrated that SingleR provides reliable annotations when appropriately matched references are available, with performance metrics that can be used to evaluate annotation quality [48]. The ScPCA Portal team, after systematic benchmarking, selected SingleR as one of their primary annotation tools based on its seamless integration with SingleCellExperiment objects, cost efficiency, and provision of quality metrics [48].

Practical Framework for Reference Dataset Selection

Criteria for Evaluating Reference Datasets

When selecting reference datasets for SingleR, researchers should consider multiple criteria to ensure optimal annotation performance. The following table summarizes the key evaluation dimensions:

Table 1: Criteria for Reference Dataset Evaluation

Criterion	Considerations	Impact on Annotation
Technology Compatibility	Platform (bulk vs. single-cell), protocol (UMI vs. full-length), normalization method	Technical biases can reduce cross-dataset comparability [8]
Biological Relevance	Tissue/organ match, species compatibility, disease state alignment	Ensures reference contains biologically similar cell types [31]
Annotation Quality	Resolution of labels, validation methods, expertise of original annotators	Determines accuracy and granularity of transferred labels [48]
Cell Type Coverage	Diversity of included cell types, presence of rare populations, lineage representation	Affects ability to identify all cell types in test data [8]
Sample Size	Number of cells/samples per label, balance across labels	Influences statistical power and robustness of scores [10]

Types of Reference Data and Their Applications

Reference datasets for SingleR generally fall into two categories with distinct characteristics and applications:

Bulk RNA-seq references (used in "classic" mode) provide well-established cell type signatures derived from purified populations. These typically have high-quality annotations but may lack resolution for closely related cell states. The ImmGen dataset, for example, contains immune cell profiles with carefully validated labels [8]. The classic mode employs a marker detection algorithm based on log-fold changes in median expression, making it suitable for references with limited replication [8].

Single-cell RNA-seq references enable like-for-like comparison with test data and can capture greater cellular heterogeneity. These references use statistical tests (e.g., Wilcoxon rank sum) for marker detection that account for cell-to-cell variation [47]. Single-cell references can be used in their native format or aggregated into "pseudo-bulk" samples to improve computational efficiency while preserving some heterogeneity information through k-means clustering within labels [47].

Source-Specific Reference Datasets

Several curated reference datasets are readily available through Bioconductor packages like celldex, which provides standardized references for common applications:

Table 2: Commonly Used Reference Datasets

Reference Name	Type	Species	Cell Types Covered	Best Applications
Human Primary Cell Atlas (HPCA)	Bulk	Human	37 main cell types, 157 subtypes	Primary cells, immune cells [10]
ImmGen	Bulk	Mouse	Comprehensive immune cells	Immunological studies [8]
Blueprint/ENCODE	Bulk	Human	Immune and stromal cells	Human tissue profiling [14]
Mouse RNA-seq	Bulk	Mouse	Various tissues	General mouse studies [14]
Database of Immune Cell Expression (DICE)	Bulk	Human	Immune cell subsets	Human immunology [14]

Protocol for Handling Missing Cell Types

Detection and Diagnosis of Missing Cell Types

The first challenge in addressing missing cell types is recognizing their presence in the test dataset. Several indicators can suggest that a test dataset contains cell types absent from the reference:

• Consistently low confidence scores across multiple cells using pruned.labels in SingleR output [8]
• Cells clustering separately in dimensionality reduction but receiving the same label as distinct populations
• Discrepancy between marker gene expression and assigned labels
• Poor alignment between experimental expectations and annotation results

The SingleR output provides several diagnostic metrics to assess annotation quality. The scores matrix contains the correlation-based scores for each cell-label combination, while delta.next captures the difference between the highest and second-highest scores, indicating confidence in the assignment [8]. The pruned.labels field uses an internal algorithm to remove labels that are unlikely to be correct, replacing them with NA [8].

Strategic Approaches for Missing Cell Types

When missing cell types are suspected, researchers can employ several strategies to improve annotations:

Reference Modification: Augment existing references with additional data containing the missing cell types. This can be done by combining multiple references or adding custom data to an existing reference structure. The SingleR() function accepts any properly formatted reference, allowing integration of public and proprietary data [47].

Hierarchical Annotation: Implement a tiered approach where initial annotation identifies broad cell classes, followed by focused analysis on heterogeneous populations using specialized references. This strategy works particularly well for identifying rare cell populations that may be absent from general references.

Marker-Based Validation: Supplement SingleR annotations with traditional marker gene analysis to identify cells with expression patterns inconsistent with their assigned labels. These cells may represent missing types requiring further investigation [48].

Custom Marker Detection: Bypass SingleR's internal marker detection by supplying custom marker lists tailored to specific biological questions or cell types of interest using the genes argument in SingleR() [47]. This approach integrates prior biological knowledge directly into the annotation process.

Experimental Protocol for Reference Evaluation and Selection

Systematic Workflow for Reference Selection

The following experimental protocol provides a structured approach for selecting and validating reference datasets:

Diagram 1: Workflow for reference dataset selection

Step 1: Define Expected Cell Types

• Compile a comprehensive list of cell types expected in the test dataset based on biological knowledge, prior literature, and experimental design
• Categorize cell types as "essential" (must be identified) and "optional" (may be present)
• Document known marker genes for expected cell types to facilitate later validation

Step 2: Identify Candidate References

• Search curated repositories (celldex, CellMarker, PanglaoDB) for potential references
• Consider both bulk and single-cell references based on availability and research questions
• Screen references for inclusion of expected cell types using available metadata

Step 3: Assess Technical Compatibility

• Evaluate platform compatibility (bulk RNA-seq, full-length scRNA-seq, UMI-based protocols)
• Check normalization methods and gene identifiers for consistency
• Assess need for data transformation (e.g., TPM for full-length data when using celldex references) [8]

Step 4: Evaluate Biological Relevance

• Confirm tissue/organ match between reference and test data
• Verify species compatibility (consider ortholog mapping if different species)
• Assess developmental/disease state alignment
• Evaluate annotation granularity relative to research needs

Step 5: Establish Controls

• Positive control: Dataset with known ground-truth annotations to verify method performance [48]
• Negative control: "Nonsense reference" with cell types not expected in test data to confirm specificity [48]
• Process controls through identical annotation pipeline

Step 6: Benchmark Performance

• Annotate positive control with each candidate reference
• Calculate accuracy metrics compared to ground truth
• Assess confidence scores across cell types
• Identify systematic errors or missing cell types

Step 7: Select Optimal Reference

• Choose reference that maximizes accuracy for positive control
• Ensure adequate coverage of essential cell types
• Verify reasonable confidence scores across cell populations
• Document selection rationale for reproducibility

Quantitative Assessment Metrics

The benchmarking step (Step 6) requires quantitative metrics to compare reference performance:

Table 3: Metrics for Reference Performance Assessment

Metric	Calculation	Interpretation
Annotation Accuracy	Percentage of cells correctly labeled in positive control	Overall reference performance [48]
Mean Confidence Score	Average of delta.next values across all cells	Higher values indicate more confident annotations [8]
Cell Type F1 Score	Harmonic mean of precision and recall for each cell type	Balanced measure for individual cell types
Pruning Rate	Percentage of cells with pruned labels (assigned NA)	High rates suggest missing cell types or poor reference match
Cluster Homogeneity	Entropy of label distribution within clusters	Measures consistency of annotations

Visualization and Decision Support Tools

Interactive Reference Selection Framework

To support the reference selection process, we propose a decision framework that incorporates both technical and biological considerations:

Diagram 2: Decision framework for reference suitability assessment

Implementation of Hierarchical Annotation Strategy

For complex annotation scenarios involving rare cell types or multiple biological compartments, a hierarchical approach can significantly improve results:

Diagram 3: Hierarchical annotation workflow

The Scientist's Toolkit: Essential Research Reagents

Table 4: Essential Tools for Reference-Based Annotation with SingleR

Tool/Category	Specific Examples	Function/Purpose
Reference Datasets	Human Primary Cell Atlas, ImmGen, Blueprint/ENCODE	Provide labeled expression data for cell type recognition [10] [8]
Software Packages	SingleR, celldex, Seurat, SingleCellExperiment	Implement annotation algorithms and data structures [18] [14]
Validation Tools	scRNAseq, scran, scater	Generate positive controls and assess annotation quality [48]
Visualization	ggplot2, pheatmap, ComplexHeatmap	Visualize annotation results and confidence metrics [14]
Benchmarking Frameworks	scRNAseq_Benchmark, scib-metrics	Compare performance across methods and references [48]

Effective reference dataset selection and robust handling of missing cell types represent fundamental challenges in reference-based cell annotation with SingleR. This protocol has outlined a comprehensive framework for addressing these challenges through systematic reference evaluation, strategic annotation approaches, and rigorous validation. The strategies presented here—including hierarchical annotation, reference modification, and quantitative benchmarking—enable researchers to maximize annotation accuracy even when facing incomplete reference coverage.

As the single-cell field continues to evolve, with new references and improved algorithms regularly emerging, the principles outlined in this protocol will remain relevant for designing biologically informed annotation workflows. By adopting these structured approaches, researchers can enhance the reliability of their cell type annotations and generate more meaningful biological insights from single-cell RNA sequencing data.

Within the broader scope of reference-based cell annotation research, handling multiple single-cell RNA sequencing (scRNA-seq) datasets efficiently is a common challenge. The standard SingleR() workflow, while robust, can become computationally expensive when the same reference dataset is used to annotate numerous target datasets. This repetitive process recalculates marker genes and constructs nearest-neighbor indices for every run, leading to significant redundancy. The trainSingleR() function addresses this bottleneck by decoupling the reference-based training phase from cell classification. This advanced configuration allows researchers to precompute a trained classifier once, which can then be rapidly applied to multiple query datasets, dramatically improving analytical throughput for multi-dataset projects. This protocol outlines the methodology for implementing preconstructed indices, detailing the workflow, providing a benchmarked performance analysis, and presenting a practical example for annotating peripheral blood mononuclear cell (PBMC) data.

Theoretical Foundation and Workflow

The trainSingleR function executes the reference-dependent components of the SingleR algorithm, which includes feature selection (identifying marker genes) and the construction of nearest-neighbor indices in rank space [49] [50]. The resulting object encapsulates all the necessary information to classify cells in a target dataset without recalculating these reference-specific elements.

A critical prerequisite for this workflow is that the gene annotation in the test dataset must be identical to or a superset of the genes used during the training step [49] [39]. Violating this condition will cause the classification to fail. The subsequent classifySingleR function performs the annotation of the test dataset using the pre-trained model, ensuring computational efficiency while yielding results identical to the standard SingleR() function [49].

The logical relationship and data flow between these steps are illustrated in the following workflow diagram.

Figure 1. Workflow for using preconstructed indices. This diagram illustrates the key steps for leveraging the trainSingleR() and classifySingleR() functions to annotate multiple datasets efficiently.

Application Protocol

Step-by-Step Methodology

This protocol uses the DICE reference dataset [49] to annotate a PBMC 3k test dataset, demonstrating the complete process from data preparation to cell type prediction.

Step 1: Load Reference and Test Datasets Begin by loading the reference dataset (e.g., DICE from the celldex package) and the target test dataset (e.g., PBMC 3k from TENxPBMCData). The reference should be a SummarizedExperiment object or a numeric matrix of log-transformed expression values [50].

Step 2: Identify Common Genes Subset both the reference and test datasets to include only the genes common to both. This is a crucial step for ensuring compatibility between the pre-trained model and the test data.

Step 3: Train the SingleR Classifier Use the trainSingleR function on the reference data, restricted to the common genes. Setting aggr.ref=TRUE accelerates future classification by aggregating the reference into pseudo-bulk profiles [49] [50].

Step 4: Classify the Test Dataset Annotate the test dataset using the pre-trained model and the classifySingleR function.

Step 5: Validate Results (Optional) Verify that the results from the two-step process are identical to those from the direct SingleR() approach.

Key Configuration Parameters

The trainSingleR function provides several parameters to customize the training process. The table below summarizes the key arguments and their functions.

Table 1: Key Parameters for the trainSingleR Function

Parameter	Type	Default	Description
genes	Character	"de"	Feature selection method: "de" (differential expression), "sd" (standard deviation), or "all" (no selection) [50].
de.method	Character	"classic"	Method for DE gene detection: "classic", "wilcox", or "t" [50].
de.n	Integer	Formula-based	Number of DE genes to use. Defaults to 500 * (2/3) ^ log2(N) where N is the number of labels [50].
aggr.ref	Logical	FALSE	Whether to aggregate reference into pseudo-bulk samples for speed [49] [50].
sd.thresh	Numeric	1	Minimum threshold on the standard deviation per gene when genes="sd" [50].
restrict	Character	NULL	Vector of gene names to restrict marker selection to [50].
BNPARAM	Object	KmknnParam()	Algorithm for building nearest-neighbor indices [49].

Performance & Benchmarking

Efficiency and Accuracy

The primary advantage of using preconstructed indices is a substantial reduction in computation time for projects involving multiple target datasets. The training step is performed once, and the saved trained object can be reused indefinitely, eliminating redundant calculations [49].

In terms of annotation accuracy, a recent independent benchmarking study evaluated five reference-based cell type annotation tools on 10x Xenium spatial transcriptomics data. The study concluded that SingleR was the best performing tool, being fast, accurate, and easy to use, with results closely matching manual annotation [35] [51]. This validates the underlying algorithm that the trainSingleR approach leverages.

Comparison with Alternative SingleR Workflows

The table below quantitatively compares the preconstructed indices strategy with other advanced configurations available in SingleR, highlighting the trade-offs between speed and annotation resolution.

Table 2: Performance Comparison of Advanced SingleR Configurations

Configuration	Relative Speed	Best Use Case	Key Advantage	Key Limitation
Preconstructed Indices (trainSingleR)	Very Fast	Annotating multiple datasets with one reference.	Eliminates redundant training calculations [49].	Test dataset genes must be a superset of training genes [49].
Cluster-Level Annotation (clusters=)	Fastest	Annotating pre-clustered data for high-level analysis.	Extremely fast; easy to interpret cluster-level identity [49] [39].	Loses single-cell resolution and masks cellular heterogeneity [39].
Approximate Algorithms (fine.tune=FALSE, BNPARAM=AnnoyParam())	Fast	Large datasets where a minor accuracy loss is acceptable.	Good speed-accuracy trade-off by skipping fine-tuning [49].	Potential reduction in annotation accuracy, especially for fine labels [49].
Parallelization (BPPARAM=MulticoreParam())	Faster (depends on cores)	Large datasets on multi-core systems (Linux/Mac).	Leverages multiple CPUs to reduce wall-clock time [49].	Limited speedup on Windows; requires SnowParam [49].

The Scientist's Toolkit

Table 3: Essential Research Reagent Solutions for SingleR Annotation

Item	Function/Description	Example Source/Bioconductor Package
Reference Datasets	Provides the labeled expression data used to train the SingleR classifier.	celldex (e.g., DICE, Human Cell Atlas, Blueprint/ENCODE) [49].
Pre-trained Model	The output of trainSingleR(), containing marker genes and precomputed indices for rapid classification.	Output of trainSingleR() function [49] [50].
High-Quality scRNA-seq Data	The unannotated test dataset(s) to be classified using the pre-trained model.	Public repositories (e.g., cellxgene) or in-house single-cell experiments.
BiocNeighbor Index	Data structure enabling fast nearest-neighbor searches during classification.	BiocNeighbors package (e.g., KmknnParam, AnnoyParam) [49].

Concluding Remarks

The use of preconstructed indices via trainSingleR and classifySingleR represents a best practice for computational efficiency in large-scale single-cell annotation projects. This methodology is particularly powerful in the context of a broader thesis on reference-based annotation, as it facilitates the consistent application of a single curated reference across multiple studies or experimental batches. By adhering to this protocol, researchers and drug development professionals can significantly accelerate their analysis pipeline while maintaining the high standard of accuracy associated with the SingleR method.

Ensuring Robust Results: Benchmarking SingleR and Comparing Methodologies

Accurate cell type annotation is a critical step in the analysis of single-cell RNA sequencing (scRNA-seq) data, forming the foundation for all subsequent biological interpretations [52] [53]. SingleR is a widely adopted and robust correlation-based tool for automated cell type identification, which assigns labels to cells by comparing their transcriptomic profiles to a well-annotated reference dataset [9] [54]. However, like any computational method, its predictions are not infallible. The reliability of its output can be influenced by factors such as the quality and completeness of the reference data, the similarity between closely related cell types, and the presence of unknown or pathological cell states [9] [3]. Therefore, employing a rigorous, multi-faceted validation strategy is not merely recommended but essential for ensuring biological accuracy. This protocol details two fundamental and powerful approaches for validating SingleR predictions: diagnostic checks internal to the SingleR workflow and external cross-referencing using marker gene expression. By integrating these methods, researchers can quantify assignment confidence, identify potentially mislabeled cells, and build a robust foundation for downstream analysis in drug development and basic research.

Validation Method I: Diagnostic Checks within SingleR

SingleR provides built-in diagnostics that help assess the confidence of each cell type assignment without requiring external data. These diagnostics primarily focus on the scores generated during the correlation-based comparison.

Based on the Scores Within Cells

The scores matrix returned by SingleR() contains the correlation-based score for each cell (row) against every reference label (column). The key diagnostic is the spread of these scores for a given cell.

• Procedure:
  • Access Scores: Extract the scores matrix from the SingleR result object (e.g., pred.grun$scores).
  • Visualize Scores: Use the plotScoreHeatmap() function to visualize the score matrix. This heatmap is designed to highlight differences between labels within each cell, making it easy to spot cells with ambiguous assignments [9].
  • Interpretation: Ideally, for each cell, one label's score should be distinctly higher than all others. Cells where multiple labels have similar scores indicate uncertain assignments. This uncertainty may be acceptable if it is between biologically similar cell types (e.g., T cell subsets) but warrants further investigation otherwise.

Based on the Deltas Across Cells

A more robust diagnostic is the "delta", defined for each cell as the difference between the score for its assigned label and the median score across all labels for that cell.

• Procedure:
  • Calculate Delta: The delta is automatically computed by SingleR. The per-cell deltas are available in the SingleR result object.
  • Thresholding: SingleR can automatically prune low-confidence assignments using an outlier-based method on the deltas, reported in the pruned.labels field (where low-confidence assignments are set to NA) [9].
  • Manual Thresholding: For more control, manually apply a threshold using the pruneScores() function. A common approach is to set a fixed minimum delta (e.g., min.diff.med = 0.2), where higher values enforce greater stringency.
  • Visualize Distribution: Use plotDeltaDistribution() to visualize the distribution of deltas across all cells or grouped by their assigned label. Labels with consistently low deltas should be treated with caution [9].

Table 1: Key Diagnostic Metrics Provided by SingleR

Metric	Description	Interpretation	How to Access
Scores Matrix	Correlation scores for each cell against every reference label.	A clear top-scoring label indicates a confident assignment.	pred$scores
Delta (Δ)	Difference between the assigned label's score and the median score across all labels for a cell.	A low delta indicates an ambiguous assignment.	Calculated internally; visualized with plotDeltaDistribution()
Pruned Labels	Labels for cells that failed the confidence threshold (set to NA).	Identifies cells for which no confident label could be assigned.	pred$pruned.labels

The following diagram illustrates the logical workflow for performing and interpreting these internal diagnostic checks.

Validation Method II: Marker Gene Overlap

A biologically intuitive and critical validation step is to verify that cells assigned to a particular label express canonical marker genes for that cell type. This serves as an external check on the SingleR prediction.

Marker Gene Expression Heatmaps

SingleR facilitates this through the plotMarkerHeatmap() function, which visualizes the expression of the most relevant marker genes in the test dataset.

• Procedure:
  • Retrieve Markers: The marker genes used by SingleR for the assignment are stored in the result's metadata.
  • Generate Heatmap: Execute plotMarkerHeatmap(pred, sce, label), where pred is the SingleR result, sce is the SingleCellExperiment object containing the test data, and label is the specific cell type to validate.
  • Interpretation: Confidently assigned cells should show strong expression of the label's marker genes. For example, beta cells should robustly express insulin (INS) [9]. If the identified markers are not meaningful or not consistently upregulated, the assignment should be questioned.
  • Batch Analysis: To systematically validate all labels, wrap the plotMarkerHeatmap() function in a loop to generate a heatmap for every cell type.

Cross-Reference with Manual Annotation & Unsupervised Clustering

Comparing SingleR assignments with other independent methods provides a powerful consensus view.

• Comparison with Unsupervised Clusters:

  • Procedure: Compare the SingleR-assigned labels to the groupings generated from an unsupervised clustering algorithm (e.g., graph-based clustering on the PCA or UMAP space of the test data) [9].
  • Interpretation: There should be strong concordance. If a SingleR label spans multiple distinct unsupervised clusters, it may indicate an under-resolved cell population. Conversely, if a single cluster is split across multiple SingleR labels, it may suggest over-fitting or the presence of a novel cell state not in the reference.

• Comparison with Manual Annotation:

  • Procedure: Use established marker genes from literature to manually annotate clusters in the test dataset. Then, construct a confusion matrix to compare these manual labels against the SingleR predictions.
  • Interpretation: High agreement, as measured by metrics like overall accuracy or Adjusted Rand Index (ARI), increases confidence. Discrepancies highlight cell populations that require careful re-examination.

Table 2: Summary of Validation Approaches and Their Applications

Validation Method	Key Function/Tool	Strengths	Best Used For
Score & Delta Diagnostics	plotScoreHeatmap(), plotDeltaDistribution(), pruneScores()	Fast, integrated into SingleR workflow, quantitative.	Initial confidence assessment, filtering out low-quality assignments.
Marker Gene Expression	plotMarkerHeatmap()	Biologically interpretable, uses test dataset's intrinsic signals.	Biological plausibility check, identifying misannotations based on known biology.
Unsupervised Clustering	FindClusters() (Seurat), clusterCells() (scran)	Data-driven, reference-free, can reveal novel subtypes.	Identifying potential over-/under-clustering and novel cell states.
Comparison with Manual	Confusion matrix, ARI	Considered a "gold standard" for known cell types.	Final benchmarking, especially when canonical markers are well-established.

The integrated workflow for marker gene validation and cross-referencing is shown below.

The Scientist's Toolkit: Essential Research Reagents & Tools

The following table catalogs key software tools and resources essential for executing the validation protocols described in this document.

Table 3: Key Research Reagent Solutions for SingleR Validation

Tool/Resource	Function	Application in Validation Protocol
SingleR (R/Bioconductor)	Correlation-based automated cell type annotation.	The core tool generating the predictions to be validated. Provides built-in diagnostics like scores and deltas [9].
scater / scran (R/Bioconductor)	Single-cell analysis toolkit for data handling, normalization, and clustering.	Used for quality control, normalization, and performing unsupervised clustering for cross-reference validation [53].
Seurat (R)	Comprehensive single-cell analysis platform.	An alternative environment for clustering, visualization (UMAP), and marker gene detection for cross-referencing [35] [54].
celldex (R/Bioconductor)	Repository of curated reference datasets.	Provides high-quality, standardized reference datasets (e.g., Human Primary Cell Atlas) for running SingleR, which is critical for obtaining accurate initial predictions [53].
AUCell / GSEABase	Gene set enrichment analysis at the single-cell level.	Can be used to quantify the activity of predefined cell type-specific gene sets, providing an additional layer of marker-based validation [53].

Validating the output of automated cell type annotation tools like SingleR is a non-negotiable step in rigorous single-cell analysis. Relying solely on the raw predictions introduces unnecessary risk and can compromise downstream biological conclusions. This application note has detailed a synergistic validation strategy combining SingleR's internal diagnostics—interrogating the scores and deltas—with external, biologically grounded checks using marker gene expression and unsupervised clustering. By systematically applying this multi-pronged protocol, researchers and drug development professionals can quantify confidence, identify and prune ambiguous assignments, and ultimately arrive at a high-fidelity annotation of their single-cell data. This robust foundation is crucial for generating reliable insights into cellular mechanisms, identifying novel drug targets, and understanding disease pathology.

Reference-based cell type annotation is a fundamental step in single-cell RNA sequencing (scRNA-seq) analysis, where tools like SingleR assign cell identities by comparing query data to expertly labeled reference datasets [9]. The exponential growth of computational methods for single-cell data analysis presents researchers with a double-edged sword: a wealth of choices alongside significant challenges in selecting appropriate methodologies [55]. Benchmarking studies serve as critical resources for navigating this complex landscape by providing systematic, empirical evaluations of method performance. A comprehensive benchmarking framework must assess three core performance metrics: accuracy (the correctness of cell type predictions), consistency (the reliability of results across variations in input or parameters), and computational efficiency (the resource consumption required to obtain results) [55]. This application note details standardized protocols for evaluating these metrics within the context of reference-based cell annotation, providing researchers with methodologies to rigorously validate tools against their specific research needs and resource constraints.

Core Performance Metrics and Evaluation Framework

Defining the Metric Triad

The performance of cell type annotation methods rests on three interdependent pillars:

• Accuracy quantifies the correctness of cell type assignments against a known ground truth. It is typically measured by the agreement between computational predictions and manual annotations by domain experts [56] [57]. For SingleR, this is reflected in the per-cell scores, where ideally one label's score is clearly larger than others [9].
• Consistency evaluates the robustness of a method's output when faced with technically inconsequential variations, such as changes in input gene order, parameter settings, or reference data subsampling [58]. A consistent method produces stable predictions across these minor perturbations.
• Computational Efficiency measures the resources required to complete annotations, including runtime, memory consumption, and scalability to large datasets [55] [59]. This determines the practical feasibility of applying a method to ever-increasing single-cell datasets.

The SingleR Workflow and Assessment Points

Table: Key Diagnostic Measures in the SingleR Workflow

Workflow Stage	Diagnostic Measure	Interpretation	Purpose in Evaluation
Scoring	Per-cell scores matrix	Ideally shows one label's score is clearly larger than others [9]	Assess assignment confidence and accuracy
Confidence Assessment	Delta (Δ)	Difference between assigned label's score and median of all scores [9]	Identify low-confidence assignments; filter uncertain calls
Quality Control	Pruned labels	Labels replaced with NA after outlier-based filtering [9]	Remove unreliable assignments that could impact accuracy
Biological Validation	Marker gene expression	Examination of canonical marker expression in assigned cells [9]	Verify biological plausibility of annotations

The following diagram illustrates the core SingleR workflow and its key evaluation points:

Experimental Protocols for Metric Evaluation

Protocol for Assessing Annotation Accuracy

Objective: Quantify the correctness of cell type predictions against experimentally validated or manually curated ground truth labels.

Materials:

• Reference datasets with validated cell type labels (e.g., from cell atlas projects) [60]
• Query datasets with known cell identities for validation
• Computing environment with SingleR installed (R/Bioconductor)
• Benchmarking scripts for accuracy calculation (available from https://github.com/SydneyBioX/scbenchbenchmark [55])

Procedure:

• Data Preparation: Obtain a reference dataset with expertly annotated cell types. For the query dataset, use either:
  • Experimental data with known cell identities validated by orthogonal methods (e.g., FACS sorting, known markers)
  • Synthetic data with predefined cell type labels [59] [61]

• Method Application: Run SingleR on the query dataset using the reference following standard parameters [9]:

• Accuracy Calculation: Compare predictions to ground truth using:

  • Overall accuracy: Proportion of correctly labeled cells across all types
  • Per-class accuracy: Precision and recall for each cell type
  • Confusion matrix: Detailed analysis of misclassification patterns

• Benchmark Comparison: Execute competing methods (e.g., Seurat, scMAP, CellTypist) on identical datasets using comparable parameters [56].

• Statistical Analysis: Apply appropriate statistical tests to determine significant performance differences between methods, accounting for multiple comparisons.

Protocol for Evaluating Consistency

Objective: Determine the robustness of annotation results to technically inconsequential variations in input data and parameters.

Materials:

• Reference dataset with well-defined cell type labels
• Query dataset of sufficient size (>1,000 cells recommended)
• High-performance computing resources for parallel processing

Procedure:

• Input Perturbation Tests:
  • Gene set variation: Run SingleR with different numbers of highly variable genes (top 500, 1,000, 2,000)
  • Reference subsampling: Execute annotations with randomly sampled subsets of reference data (80%, 60%, 40% of cells)
  • Label shuffling: Test robustness to minor changes in reference cell type labels

• Parameter Sensitivity Analysis:

  • Fine-tuning parameters: Evaluate consistency across different fine-tuning settings
  • Threshold variations: Test impact of different pruning thresholds on final assignments

• Consistency Quantification:

  • Metric: Calculate consistency rate (CR) as the proportion of cells receiving identical labels across perturbations [58]
  • Analysis: Compute Cohen's κ between annotations from different conditions to measure agreement beyond chance [57]

• Visualization: Generate plots showing:

  • Distribution of label changes across perturbation conditions
  • Heatmaps of assignment stability across cell types

Protocol for Measuring Computational Efficiency

Objective: Quantify the computational resources required for cell type annotation and determine scalability to large datasets.

Materials:

• Computing infrastructure with resource monitoring capabilities
• Datasets of varying sizes (from 1,000 to 1,000,000 cells)
• System monitoring tools (e.g., Linux time command, R bench package)

Procedure:

• Experimental Setup:
  • Prepare datasets of increasing sizes (e.g., 1K, 5K, 10K, 50K, 100K cells)
  • Standardize computing environment (CPU, memory, storage type)

• Resource Monitoring:

  • Execute SingleR on each dataset size with 3 replicates
  • Record wall-clock time, CPU time, and peak memory usage
  • Monitor disk I/O and temporary storage requirements

• Scalability Analysis:

  • Fit computational complexity curves (linear, polynomial, exponential) to time/memory vs. cell count data
  • Identify breaking points where resource demands become prohibitive

• Comparative Benchmarking:

  • Execute multiple methods on identical hardware and datasets
  • Rank methods by efficiency while controlling for accuracy [59]

Benchmarking Results and Interpretation

Quantitative Performance Comparisons

Table: Comparative Performance of Cell Type Annotation Methods

Method	Overall Accuracy	Consistency Rate	Runtime (10K cells)	Memory Usage	Key Strengths
SingleR	0.82-0.89 [56]	0.85-0.92 [57]	Medium	Medium	Excellent balance of accuracy and speed [56]
Seurat	0.85-0.91 [56]	0.83-0.90	Fast	Low	Best for major cell types [56]
scMAP	0.78-0.84	0.80-0.87	Fast	Low	Rapid annotations for initial screening
CellTypist	0.80-0.86	0.82-0.88	Medium	Medium	Good for immune cell subsets
GPT-4	0.79-0.88 [57]	0.85 (exact match) [57]	Slow (API dependent)	Low	Contextual understanding of markers [57]

Interpreting Diagnostic Information

The relationship between SingleR's diagnostic outputs and final annotation quality can be visualized as follows:

Key Interpretation Guidelines:

• Score Matrix: Examine the spread of scores within each cell. Similar scores for multiple labels indicate uncertain assignments, though this may be acceptable for closely related cell types [9].
• Delta Values: Low deltas indicate uncertain assignments, possibly because the cell's true label is missing from the reference. Use plotDeltaDistribution() to visualize per-label delta distributions [9].
• Pruning Impact: The default pruning parameters may not be appropriate for every dataset. Manually call pruneScores() with custom min.diff.med values for dataset-specific filtering [9].
• Marker Validation: Use plotMarkerHeatmap() to verify that cells assigned to a label strongly express that label's canonical markers. This provides biological plausibility for annotations [9].

The Scientist's Toolkit

Essential Research Reagent Solutions

Table: Key Reagents and Resources for Benchmarking Studies

Resource Category	Specific Examples	Function in Benchmarking	Availability
Reference Datasets	Human Cell Atlas, Tabula Sapiens, Tabula Muris	Provide gold-standard labels for accuracy assessment [60] [57]	Public portals and repositories
Annotation Tools	SingleR, Seurat, scMAP, CellTypist	Methods under evaluation [9] [11] [56]	Bioconductor, CRAN, GitHub
Benchmarking Frameworks	SimBench, SCORE principles [58] [61]	Provide standardized evaluation metrics and workflows	GitHub repositories
Synthetic Data Generators	SPARSim, ZINB-WaVE, SymSim	Create data with known ground truth for controlled testing [59] [61]	R/Bioconductor packages
Experimental Validation Sets	Cell hashing, Species mixing, MULTI-seq	Provide experimental doublet detection for validation [59]	Specialized protocols

This application note presents comprehensive protocols for benchmarking the performance of reference-based cell type annotation methods, with emphasis on SingleR. The structured assessment of accuracy, consistency, and computational efficiency enables researchers to make evidence-based method selections suited to their specific research contexts and resource constraints.

Based on current benchmarking evidence [55] [56], SingleR provides an excellent balance of accuracy and interpretability for standard annotation tasks, while Seurat performs well for major cell type identification. For projects with limited computational resources, scMAP offers a faster alternative with slightly reduced accuracy. Emerging approaches like GPT-4 show promise for leveraging contextual knowledge from published literature but require validation against experimental data [57].

Benchmarking studies consistently reveal that no single method outperforms all others across all metrics and datasets [55] [61]. Researchers should therefore select methods based on their specific priorities—whether accuracy, speed, or robustness—and employ the protocols outlined here to validate performance for their particular use cases. As the single-cell field continues to evolve with new methods and larger datasets, rigorous benchmarking remains essential for navigating the complex landscape of computational tools and ensuring biologically meaningful research outcomes.

Cell type annotation is a critical, indispensable step in the analysis of single-cell RNA sequencing (scRNA-seq) data, forming the foundation for understanding cellular composition and function in health and disease [3] [62]. This field is currently characterized by two competing methodological paradigms: established reference-based approaches and emerging artificial intelligence (AI)-driven methods. Reference-based tools, exemplified by SingleR, operate by comparing query scRNA-seq data to curated reference datasets of pure cell types, transferring labels based on expression similarity [18] [62]. In contrast, a new generation of annotation tools leverages the power of large language models (LLMs) to interpret cell identity directly from marker gene information. These methods, including the recently developed LICT (Large language model-based Identifier for Cell Types) and scExtract, aim to replicate and scale expert reasoning without direct dependency on reference data [3] [13]. This application note provides a detailed comparative analysis of these approaches, offering structured performance data, experimental protocols, and practical guidance for researchers navigating this evolving landscape.

SingleR: Reference-Based Annotation

SingleR is a computational method designed for unbiased cell type recognition in scRNA-seq data. Its core methodology leverages reference transcriptomic datasets of pure cell types to independently infer the cell of origin for each single cell within a query dataset. Unlike methods that rely heavily on known marker genes and manual cluster annotation, which can introduce subjectivity and limit the differentiation of closely related cell subsets, SingleR provides an automated, data-driven approach. After processing data with analysis packages like Seurat, SingleR's annotations can be used for downstream analysis and visualization, offering a powerful, integrated tool for scRNA-seq investigation [18] [62].

LICT and scExtract: The LLM-Based Paradigm

Emerging LLM-based tools represent a significant shift in annotation strategy. LICT (Large language model-based Identifier for Cell Types) is a software package that employs a multi-model integration and a "talk-to-machine" strategy. It was developed to address the limitations of both expert-driven and automated methods, which can be biased or constrained by their training data. LICT does not rely on reference datasets, which enhances its generalizability and helps prevent errors that require time-consuming corrections [3] [63].

Another tool in this space, scExtract, is a framework that leverages LLMs to fully automate scRNA-seq data analysis, from preprocessing to annotation and prior-informed multi-dataset integration. It extracts critical information, such as filtering parameters and marker gene descriptions, directly from research articles to guide data processing in a manner that aligns with the original authors' methodology. This approach has been shown to outperform existing reference transfer methods in benchmarks and enables the creation of integrated cell atlases by incorporating prior annotation information for improved batch correction [13].

Table 1: Core Methodological Differences Between SingleR and LLM-Based Tools.

Feature	SingleR (Reference-Based)	LICT/scExtract (LLM-Based)
Core Principle	Compares cells to a reference dataset of pure cell types [62].	Uses LLMs to interpret marker genes and article context [3] [13].
Dependency	Requires high-quality, comprehensive reference data [62].	Reference-free; relies on the embedded knowledge of multiple LLMs [3].
Automation Level	Automates label transfer after reference is set.	High; can automate from raw data to annotation, including parameter extraction [13].
Handling Novelty	Limited to cell types present in the reference.	Potentially better at identifying novel/rare cell types not in reference datasets [13].
Key Innovation	Unbiased, data-driven label transfer.	Multi-model fusion & "talk-to-machine" for reliability assessment [3] [63].

Performance Benchmarking and Quantitative Comparison

Performance of LLM-Based Annotation

The development of LICT involved a systematic evaluation of 77 publicly available LLMs on a benchmark PBMC dataset. Five top-performing models were integrated: GPT-4, LLaMA-3, Claude 3, Gemini, and the Chinese model ERNIE 4.0 [3]. This multi-model integration strategy proved crucial for improving annotation accuracy, especially in challenging datasets with low cellular heterogeneity.

• High-Heterogeneity Datasets: In highly heterogeneous datasets like PBMCs and gastric cancer samples, LICT's multi-model integration significantly reduced the mismatch rate with manual annotations compared to a predecessor tool, GPTCelltype. For PBMC data, the mismatch rate dropped from 21.5% to 9.7%, and for gastric cancer data, from 11.1% to 8.3% [3].
• Low-Heterogeneity Datasets: Performance challenges were noted in low-heterogeneity environments (e.g., human embryo and stromal cells). However, LICT's "talk-to-machine" strategy—an iterative process that validates initial annotations by checking marker gene expression and re-querying the model with additional data—dramatically improved results. For embryo data, the full match rate with manual annotations improved by 16-fold compared to using GPT-4 alone, reaching 48.5% [3].
• Objective Reliability Evaluation: A key innovation of LICT is its objective framework for assessing annotation credibility. This strategy evaluates the reliability of an annotation based on whether more than four marker genes identified by the LLM are expressed in at least 80% of cells within the cluster. In some low-heterogeneity datasets, LICT's annotations were deemed more credible than the original manual expert annotations [3].

Comparative Performance with Other Tools

Independent benchmarking provides context for how these tools perform against a wider field. A comprehensive evaluation of 28 single-cell clustering algorithms across various metrics highlighted several top performers like scDCC, scAIDE, and FlowSOM [64]. In a separate evaluation focused on annotation accuracy, the LLM-based tool scExtract was compared against three established methods, including SingleR. The study concluded that scExtract demonstrated higher accuracy, surpassing established methods across various tissues [13].

Table 2: Summary of Key Performance Metrics from Recent Studies.

Tool / Category	Reported Performance Advantage	Context / Dataset
LICT	Reduced mismatch rate to 9.7% (from 21.5%) [3].	PBMC data (High-heterogeneity) vs. GPTCelltype.
LICT	Increased full match rate to 48.5% (16-fold improvement) [3].	Human embryo data (Low-heterogeneity) vs. GPT-4 alone.
scExtract	Higher accuracy than established methods including SingleR [13].	Evaluation across multiple human tissues (e.g., liver, kidney).
Top Clustering Tools (e.g., scDCC, scAIDE)	Top performance in ARI, NMI on transcriptomic & proteomic data [64].	Benchmarking on 10 paired transcriptomic and proteomic datasets.

Experimental Protocols

Protocol for Cell Annotation with SingleR

The following protocol describes the standard workflow for using SingleR for cell type annotation in an R environment, typically integrated with the Seurat package.

Step-by-Step Procedure:

• Data Preprocessing: Begin with a count matrix (from a file, 10X directory, or matrix object). Create a Seurat object and perform standard QC filtering based on parameters like min.genes and min.cells [18].
• Reference Selection: Identify and load a suitable reference dataset containing transcriptomic profiles of pure cell types that are biologically relevant to your query dataset.
• Create SingleR Object: Use the CreateSinglerSeuratObject wrapper function to generate a SingleR object. This function requires the count matrix, a cell type annotation file for the reference, a project name, and specifications for technology and species [18].
• Annotation Transfer: Execute the core SingleR algorithm. The function will compare each cell in the query dataset to the reference dataset and assign a cell type label based on similarity.
• Fine-Tuning (Optional): For large datasets (e.g., >100,000 cells), the built-in fine-tuning process may be computationally intensive. The recommended workaround is to run SingleR on subsets of the data and combine the results [18].
• Integration and Visualization: Transfer the SingleR-derived labels back into the Seurat object for downstream analysis, such as dimensional reduction (t-SNE, UMAP) and visualization. The results can also be uploaded to the SingleR web application for further exploration [18].

Protocol for Cell Annotation with LICT

LICT employs a more interactive, iterative workflow centered around its "talk-to-machine" strategy, which can be implemented via its dedicated R package [3] [65].

Step-by-Step Procedure:

• Input Preparation: For each cell cluster, compile a list of top differentially expressed genes (DEGs). These will serve as the primary input for the LLMs.
• Multi-Model Query: Submit the DEG lists to the suite of integrated LLMs (e.g., GPT-4, Claude 3, Gemini) using standardized prompts to obtain initial, independent cell type annotations from each model.
• Initial Annotation Integration: Apply LICT's multi-model integration strategy to select the most consistent and confident annotation from the different LLM outputs, leveraging their complementary strengths.
• Marker Gene Retrieval & Validation ("Talk-to-Machine"): a. Query the LLM to provide a list of representative marker genes for its predicted cell type. b. Evaluate the expression of these marker genes within the corresponding cell cluster in your dataset. c. Validation Check: If more than four marker genes are expressed in ≥80% of cells, the annotation is considered validated. If not, proceed to the next step [3].
• Iterative Feedback: For annotations that fail validation, generate a structured feedback prompt that includes the failed validation results and a broader set of DEGs from the cluster. Re-submit this prompt to the LLM to prompt a revised or confirmed annotation.
• Credibility Assessment: Finally, use the marker gene expression criteria from Step 4c as an objective measure to assign a reliability score to the final annotation, independent of manual labels.

Workflow Visualization

The diagram below illustrates the core procedural workflows for both SingleR and LICT, highlighting their fundamental differences in data flow and strategy.

Successful cell type annotation, regardless of the computational method, relies on a foundation of high-quality data and biological knowledge. The following table lists key resources and their functions in this field.

Table 3: Essential Resources for Cell Type Annotation Research.

Resource Name	Type	Primary Function in Annotation
Seurat [18]	Software Package (R)	A comprehensive toolkit for single-cell genomics data preprocessing, normalization, clustering, and visualization. Often used to prepare data for SingleR.
Cellxgene [13]	Curated Database	A crowdsourced platform hosting a massive collection of publicly available, curated single-cell datasets. Useful for finding reference data and benchmarking.
ACT (Annotation of cell types) [66]	Web Server / Knowledge Base	Provides a hierarchically organized marker map built from thousands of publications. Uses the WISE method to annotate cell types from a simple gene list.
scanpy [13]	Software Package (Python)	The standard Python framework for single-cell data analysis, used by scExtract for its computational pipeline (cell filtering, clustering, etc.).
Robust Rank Aggregation (RRA) [66]	Computational Method	Used in knowledgebase construction (e.g., for ACT) to aggregate gene ranks from multiple studies, creating a robust, integrated list of cell-type markers.
Adjusted Rand Index (ARI) [64]	Benchmark Metric	A metric for quantifying clustering quality by comparing predicted and ground truth labels. Values closer to 1 indicate better performance.
Normalized Mutual Information (NMI) [64]	Benchmark Metric	Measures the mutual information between clustering results and ground truth, normalized to [0, 1]. Values closer to 1 indicate better performance.

The emergence of LLM-based tools like LICT and scExtract represents a significant evolution in the field of cell type annotation. While reference-based methods like SingleR provide an unbiased and data-driven approach, their effectiveness is inherently bounded by the quality and completeness of existing reference data. LLM-based methods offer a promising, complementary pathway by leveraging vast biological knowledge encoded in language models, providing greater independence from references and introducing novel frameworks for objectively evaluating annotation reliability.

Current evidence suggests that a hybrid or context-dependent strategy may be optimal. For well-established cell types in tissues with robust reference atlases, SingleR remains a reliable and efficient choice. However, for exploratory research involving novel, rare, or poorly characterized cell states, or for the automated, large-scale processing of public datasets, LLM-based tools like LICT and scExtract show distinct advantages. As these AI-driven methods continue to mature, they are poised to enhance reproducibility, minimize subjective biases, and accelerate the extraction of biological insight from the ever-growing volume of single-cell data.

Reference-based cell annotation is a critical step in single-cell RNA sequencing (scRNA-seq) analysis, enabling researchers to decipher cellular heterogeneity within complex tissues. SingleR has emerged as a prominent method for this task, utilizing a correlation-based approach to compare single-cell expression profiles against expertly curated reference datasets. While its core algorithm is well-established, a key question for researchers and drug development professionals is how robustly this performance generalizes across the diverse tissue environments and pathological states encountered in real-world research. This application note synthesizes current evidence to address this question, providing a comparative analysis of SingleR's performance on diverse tissues and disease states, complemented by detailed protocols for implementation and validation.

Performance Evaluation Across Technologies and Tissues

Platform Performance in Complex Tissues

The accuracy of cell type annotation begins with the quality of the underlying scRNA-seq data, which varies across experimental platforms. A systematic comparison of two high-throughput 3'-scRNAseq platforms—10× Chromium and BD Rhapsody—in complex tumour tissues revealed important performance differentials that can influence annotation quality. The study employed metrics including gene sensitivity, mitochondrial content, reproducibility, clustering capabilities, cell type representation, and ambient RNA contamination [67] [68].

Table 1: Performance Comparison of scRNA-seq Platforms in Complex Tissues

Performance Metric	10× Chromium	BD Rhapsody	Impact on SingleR Annotation
Gene Sensitivity	Similar to BD Rhapsody	Similar to 10× Chromium	Comparable gene detection provides similar reference correlation potential
Mitochondrial Content	Lower	Highest	Higher content may reflect cell stress, potentially affecting annotation
Cell Type Detection Bias	Lower sensitivity in granulocytes	Lower proportion of endothelial and myofibroblasts	Can introduce systematic annotation biases for specific cell populations
Ambient RNA Source	Droplet-based profile	Plate-based profile	Different noise patterns may affect correlation scores with reference data

These findings demonstrate that platform selection introduces specific technical biases that propagate through the analysis pipeline. SingleR's correlation-based algorithm remains susceptible to these input data characteristics, particularly the cell type detection biases observed [67]. Researchers should consider these platform-specific performance characteristics when designing experiments and interpreting SingleR annotations, especially for cell types known to be affected by these biases.

Performance in Spatial Transcriptomics Data

The extension of SingleR to emerging spatial transcriptomics technologies presents additional challenges due to substantially smaller gene panels. A recent benchmark study evaluated five reference-based cell type annotation methods on 10x Xenium data from human HER2+ breast cancer, comparing them to manual annotation based on marker genes [51] [35].

Table 2: Method Performance on 10x Xenium Spatial Transcriptomics Data

Annotation Method	Accuracy vs. Manual Annotation	Speed	Ease of Use	Suitability for Spatial Data
SingleR	Closest match	Fast	Easy	Excellent
Azimuth	Lower than SingleR	Moderate	Moderate	Good
RCTD	Lower than SingleR	Slow	Complex	Moderate
scPred	Lower than SingleR	Moderate	Moderate	Moderate
scmapCell	Lower than SingleR	Fast	Easy	Moderate

The study concluded that SingleR was the best-performing reference-based cell type annotation tool for the Xenium platform, being fast, accurate, and easy to use, with results closely matching manual annotation [35]. This demonstrates SingleR's robustness even with the limited gene sets characteristic of imaging-based spatial transcriptomics technologies, making it particularly valuable for integrating spatial context with cell identity in complex disease tissues.

The SingleR Algorithm and Diagnostic Framework

Core Methodology

SingleR's annotation process is based on correlating gene expression profiles of single cells with those of pure cell types from reference datasets. The algorithm proceeds through these stages:

• Correlation Calculation: For each single cell, Spearman correlation coefficients are computed against each sample in the reference dataset, using only variable genes from the reference [6].
• Score Aggregation: Multiple correlation coefficients per cell type are aggregated to a single value per cell type per single cell using the 80th percentile of correlation values, reducing misclassification from reference heterogeneity [6].
• Fine-Tuning: The top cell types undergo repeated correlation analysis using genes variable between them, iteratively removing the lowest-scoring cell type until only two remain, with the top value assigned as the final label [6].

Diagnostic and Validation Framework

Robust implementation requires systematic validation of annotation quality. SingleR provides multiple diagnostic approaches to assess assignment confidence, detailed in the Bioconductor SingleR book [9].

1. Score-Based Diagnostics: The scores matrix contains pre-tuned correlation scores for each cell and reference label. The plotScoreHeatmap() function visualizes this matrix, where unambiguous assignments show one label with clearly higher scores than others. Clusters of cells with similar scores across multiple labels indicate uncertain assignments, though this may be acceptable for closely related cell types [9].

2. Delta-Based Quality Control: The "delta" represents the difference between the assigned label's score and the median across all labels for each cell. Low deltas indicate uncertain assignments, potentially because the cell's true type is absent from the reference. SingleR implements automated pruning using an outlier-based strategy on these deltas, reported in the pruned.labels field. The plotDeltaDistribution() function visualizes per-label delta distributions for quality assessment [9].

3. Marker Gene Validation: Expression of marker genes for assigned labels provides biological validation. The plotMarkerHeatmap() function visualizes expression of the most relevant markers—those upregulated in the test dataset and responsible for driving classification. Confident assignments should show strong expression of appropriate markers (e.g., insulin expression in beta cells) [9].

Experimental Protocol for SingleR Annotation

Reference Dataset Preparation

Materials:

• High-quality reference dataset (e.g., ImmGen for mouse, Blueprint/Encode for human)
• Computing environment with R/Bioconductor
• SingleR package installed

Procedure:

• Reference Selection: Choose a reference dataset appropriate for your biological system. For mouse studies, the Immunological Genome Project (ImmGen) database provides 830 microarray samples across 20 main cell types (253 subtypes). Alternatively, use a dataset of 358 mouse RNA-seq samples across 28 cell types. For human samples, use Blueprint Epigenomics (144 RNA-seq pure immune samples across 28 types) or Encode (115 RNA-seq stroma and immune samples across 17 types) [6].
• Quality Control: Remove low-quality cells and potential doublets from the reference using tools like scDblFinder to improve reference purity [35].
• Data Preprocessing: Normalize the reference data using appropriate methods (e.g., NormalizeData in Seurat). Select highly variable genes and scale the data [35].

Query Data Processing

Procedure:

• Quality Control: Filter out low-quality cells based on mitochondrial percentage, gene counts, and other quality metrics. For Xenium data, remove cells annotated as "Unlabeled" [35].
• Normalization: Normalize query data using the NormalizeData function. For limited gene panels (e.g., Xenium), use all genes instead of selecting highly variable genes [35].
• Data Scaling: Scale the query data using the ScaleData function in preparation for correlation analysis [35].

Cell Type Annotation Execution

Procedure:

• Run SingleR: Execute the SingleR() function with the prepared reference and query datasets. The function returns predicted labels and diagnostic information [9] [35].
• Fine-Tuning: By default, SingleR performs fine-tuning to distinguish closely related cell types. This step uses an increasing number of variable genes (decreasing N) with each iteration to improve discrimination [6].
• Confidence Assessment: Examine the scores matrix and delta values to identify low-confidence assignments. Use plotScoreHeatmap() and plotDeltaDistribution() for visualization [9].
• Pruning: Apply pruning to remove low-confidence assignments using the pruneScores() function, either with the default outlier-based method or a fixed threshold via the min.diff.med parameter [9].

Validation and Interpretation

Procedure:

• Marker Gene Verification: Use plotMarkerHeatmap() to visualize expression of canonical markers for assigned labels. Verify that cells express appropriate markers for their assigned type [9].
• Comparison with Clustering: Compare SingleR assignments with unsupervised clustering results to identify potential misannotations or novel cell types not present in the reference [9].
• Biological Consistency: Assess whether annotations make biological sense in context, such as checking for appropriate spatial distributions in spatial transcriptomics data or reasonable cell type proportions for the tissue being studied [35].

Table 3: Key Research Reagents and Computational Resources

Resource Type	Specific Examples	Function/Application
Reference Datasets	ImmGen (mouse), Blueprint Epigenomics (human), Encode (human)	Provide purified cell type expression profiles for correlation-based annotation
Single-Cell Platforms	10x Chromium, BD Rhapsody, 10x Xenium	Generate single-cell or spatial transcriptomics input data for annotation
Analysis Packages	SingleR (Bioconductor), Seurat, Azimuth	Perform cell annotation, data normalization, and quality control
Quality Control Tools	scDblFinder, InferCNV	Identify doublets in reference data and assess copy number variations in tumor cells

This comparative analysis demonstrates that SingleR maintains robust performance across diverse experimental contexts, including complex tissues and emerging spatial transcriptomics technologies. Its correlation-based approach, complemented by fine-tuning and comprehensive diagnostic capabilities, makes it particularly valuable for drug development and research applications where accurate cell type identification is crucial for understanding disease mechanisms and treatment effects.

The platform-specific biases identified in scRNA-seq technologies highlight the importance of considering experimental design when planning SingleR annotations. Meanwhile, SingleR's superior performance with Xenium spatial data positions it as a key tool for integrating cellular identity with spatial context in tissue microenvironments—particularly valuable for cancer research and characterizing complex disease states.

The diagnostic framework provided enables researchers to assess annotation confidence and identify potentially problematic assignments, while the standardized protocols facilitate reproducible implementation across diverse research projects. As single-cell technologies continue to evolve, SingleR's reference-based approach provides a flexible framework for cell type annotation that can incorporate increasingly sophisticated reference datasets, enhancing its utility for characterizing cellular heterogeneity in health and disease.

Accurate cell type annotation represents a critical bottleneck in single-cell RNA sequencing (scRNA-seq) analysis, with implications spanning basic research to drug development. Traditional approaches—whether manual expert annotation or automated reference-based methods—suffer from significant limitations including subjectivity, reference bias, and limited reproducibility [3] [69]. The SingleR package addresses these challenges by providing a computational framework for automated cell type annotation using well-curated reference datasets [5]. However, like all annotation methods, its results require rigorous validation to establish biological credibility. This application note presents an objective framework for assessing annotation reliability within the context of SingleR-based workflows, enabling researchers to distinguish high-confidence assignments from potentially spurious results and thereby enhance the rigor of downstream analyses in therapeutic development pipelines.

Quantitative Diagnostics for Annotation Quality

SingleR generates multiple quantitative diagnostics that enable researchers to evaluate annotation confidence at single-cell resolution. These metrics provide complementary perspectives on assignment quality and form the foundation of a comprehensive reliability assessment framework [9].

Table 1: Key Diagnostic Metrics Provided by SingleR

Diagnostic Metric	Calculation	Interpretation	Threshold Guidelines
Per-cell Scores	Correlation between cell and reference profiles	Higher scores indicate stronger similarity to reference	Scores should be examined relative to other labels rather than as absolute values
Delta (Δ)	Difference between assigned label score and median across all labels	Measures annotation confidence; higher Δ indicates unambiguous assignment	Default: outlier-based pruning; Conservative: Δ > 0.2 [9]
Fine-tuning Delta	Difference between highest and second-highest scores after fine-tuning	Identifies cells with distinct identities, even among closely related types	Conservative filter that may exclude biologically similar cell types
Pruned Labels	Automated filtering of low-confidence assignments	Replaces uncertain annotations with NA	Based on outlier detection within cell type groups

The delta (Δ) metric is particularly valuable for identifying ambiguous assignments that may represent unknown cell states, doublets, or low-quality cells [9]. Systematic analysis of delta distributions across cell populations reveals annotation robustness, with higher deltas indicating confident assignments and lower deltas signaling potential issues requiring further investigation.

Implementation Protocol: Reliability Assessment Workflow

Computational Requirements and Setup

Table 2: Essential Research Reagent Solutions for SingleR Annotation

Resource Type	Specific Examples	Function/Purpose	Availability
Reference Data	HumanPrimaryCellAtlasData, ImmGen data, "Th-Express" mouse CD4+ T cell atlas	Provides curated expression profiles with validated cell type labels	celldex package, custom datasets [31] [14]
Software Packages	SingleR, celldex, Seurat, scater, BiocStyle	Enables annotation execution, visualization, and diagnostic assessment	Bioconductor, CRAN [5] [14]
Visualization Tools	plotScoreHeatmap(), plotDeltaDistribution(), plotMarkerHeatmap()	Facilitates diagnostic interpretation and quality assessment	SingleR package [9]

Step-by-Step Reliability Assessment Protocol

• Annotation Execution: Process single-cell data using SingleR with appropriate reference dataset. The choice of reference significantly impacts results; blood-derived samples should use hematopoiesis-focused references while neural tissues require brain-specific atlases [31] [14].

• Score Visualization: Generate a heatmap of assignment scores to identify uncertain annotations where multiple labels show similar correlation values.

• Delta Analysis: Calculate and examine delta distributions to identify low-confidence assignments.

• Marker Expression Validation: Verify biological plausibility by examining expression of canonical marker genes for assigned labels.

• Comparison with Unsupervised Clustering: Integrate annotation results with unsupervised clustering to identify potential discrepancies that may reveal novel cell states or annotation errors.

Figure 1: Comprehensive workflow for assessing annotation reliability in SingleR, integrating multiple diagnostic approaches.

Advanced Reliability Assessment Strategies

Multi-Model Integration and LLM-Based Verification

Recent advancements in cell type annotation include the development of LLM-based tools like LICT (Large Language Model-based Identifier for Cell Types), which provides a reference-free approach to validate SingleR annotations [3]. The integration of multiple assessment models significantly enhances reliability through:

• Multi-model integration: Combining annotations from top-performing LLMs (GPT-4, Claude 3, Gemini) to leverage complementary strengths and reduce individual model uncertainties [3].
• "Talk-to-machine" iterative refinement: Implementing human-computer interaction loops where initial annotations are validated against marker gene expression, with iterative feedback improving accuracy, particularly for low-heterogeneity cell populations [3].
• Objective credibility evaluation: Establishing quantitative thresholds where annotations are considered reliable when >4 marker genes are expressed in ≥80% of cells within a cluster, providing a biologically-grounded validation metric [3].

Special Considerations for Challenging Scenarios

Table 3: Troubleshooting Annotation Reliability Issues

Challenge Scenario	Diagnostic Pattern	Recommended Resolution
Low-Heterogeneity Cell Populations	Reduced delta values, inconsistent assignments	Apply multi-model integration; Implement "talk-to-machine" iterative refinement; Utilize specialized references [3]
Closely Related Cell Subtypes	Small fine-tuning deltas, similar score profiles	Examine high-resolution marker genes; Consider hierarchical annotation; Apply conservative fine-tuning delta thresholds [9]
Novel or Unknown Cell Types	Uniformly low scores across all references, moderate deltas	Prune ambiguous assignments; Characterize as "unknown" for further investigation; Compare with unsupervised clustering [9]
Batch Effects or Technical Variability	Batch-specific annotation patterns, reduced scores	Apply appropriate integration methods; Examine batch contribution to scores; Consider within-batch normalization [9]

The establishment of an objective framework for annotation reliability represents a crucial advancement in single-cell genomics, particularly for therapeutic development where accurate cell type identification can directly impact target discovery and validation. The integrated approach presented here—combining SingleR's inherent diagnostics with multi-model verification and marker expression validation—provides a robust foundation for distinguishing confident annotations from speculative assignments.

The quantitative nature of this framework addresses a critical need in the field, where subjective assessment has traditionally introduced variability and hindered reproducibility [3] [56]. By implementing standardized reliability metrics and validation protocols, researchers can significantly enhance the credibility of their findings, particularly when investigating novel cellular targets or characterizing disease-associated cell states.

Future developments in annotation reliability will likely incorporate multimodal data integration, leveraging simultaneous measurements of gene expression, chromatin accessibility, and protein abundance to further refine cell identity assignments [70]. Additionally, the emergence of large-scale, disease-specific reference atlases will provide increasingly relevant benchmarks for annotation in therapeutic contexts, enabling more precise identification of pathological cell states targeted by investigational drugs.

For drug development professionals and translational researchers, adopting this rigorous framework for annotation reliability provides the necessary foundation for target validation, biomarker identification, and ultimately, the development of more precise therapeutic interventions targeting specific cellular populations in complex diseases.

Conclusion

SingleR establishes itself as a powerful, accessible, and reliable tool for automated cell type annotation, effectively transferring expert knowledge from curated references to novel single-cell datasets. By mastering its workflow—from foundational principles and methodological application to troubleshooting and rigorous validation—researchers can achieve high-confidence annotations that are both reproducible and scalable. Looking forward, the integration of SingleR with emerging technologies like large language models and the continuous expansion of high-quality reference datasets will further enhance its precision. This progress promises to unlock deeper biological insights in complex fields such as tumor microenvironments, developmental biology, and personalized medicine, ultimately accelerating the translation of single-cell genomics into clinical impact.

References

• 1. ScInfeR: an efficient method for annotating cell types and ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC12140018/]
• 2. Tutorial: guidelines for the experimental design of single- ... [https://www.nature.com/articles/s41596-018-0073-y]
• 3. Evaluation of cell type annotation reliability using a large ... [https://www.nature.com/articles/s42003-025-08745-x]
• 4. singleCellBase: a high-quality manually curated database of ... [https://biomarkerres.biomedcentral.com/articles/10.1186/s40364-023-00523-3]
• 5. Assigning cell types with SingleR [https://bioconductor.org/books/SingleRBook]
• 6. Cell Type Annotation of Single Cells (SingleR) [https://bio-protocol.org/exchange/minidetail?id=6646342&type=30]
• 7. SingleR - Stereopy - Read the Docs [https://stereopy.readthedocs.io/en/latest/Tutorials/SingleR.html]
• 8. Chapter 2 Using the classic mode | Assigning cell types ... [https://bioconductor.org/books/release/SingleRBook/classic-mode.html]
• 9. Chapter 4 Annotation diagnostics | Assigning cell types ... [https://bioconductor.org/books/release/SingleRBook/annotation-diagnostics.html]
• 10. Chapter 1 Introduction | Assigning cell types with SingleR [https://bioconductor.org/books/release/SingleRBook/introduction.html]
• 11. The Power of automated cell type annotation [https://singleron.bio/the-power-of-accurate-automated-cell-type-annotation/]
• 12. Using SingleR to annotate single-cell RNA-seq data [https://bioconductor.statistik.tu-dortmund.de/packages/3.11/bioc/vignettes/SingleR/inst/doc/SingleR.html]
• 13. scExtract: leveraging large language models for fully ... [https://genomebiology.biomedcentral.com/articles/10.1186/s13059-025-03639-x]
• 14. SingleR tutorial: easy cell type annotation [https://biostatsquid.com/singler-tutorial/]
• 15. Data analysis guidelines for single-cell RNA-seq in ... [https://mmrjournal.biomedcentral.com/articles/10.1186/s40779-022-00434-8]
• 16. Human Primary Cell Atlas (HPCA) reference dataset for ... [https://rdrr.io/github/dviraran/SingleR/man/hpca.html]
• 17. SingleR: Reference-Based Single-Cell RNA-Seq Annotation [https://bioc.r-universe.dev/SingleR]
• 18. SingleR: Single-cell RNA-seq cell types Recognition ( ... [https://github.com/dviraran/SingleR]
• 19. Using SingleR to annotate single-cell RNA-seq data [https://mirror.accum.se/mirror/bioconductor.org/packages/3.12/bioc/vignettes/SingleR/inst/doc/SingleR.html]
• 20. Chapter 1 Introduction | Assigning cell types with SingleR [https://bioconductor.org/books/3.17/SingleRBook/introduction.html]
• 21. Applications of single-cell RNA sequencing in drug ... [https://www.nature.com/articles/s41573-023-00688-4]
• 22. Using SingleR to annotate single-cell RNA-seq data [https://bioconductor.statistik.tu-dortmund.de/packages/3.10/bioc/vignettes/SingleR/inst/doc/SingleR.html]
• 23. Differential correlation for sequencing data - PubMed Central [https://pmc.ncbi.nlm.nih.gov/articles/PMC5244536/]
• 24. SingleR fine-tuning function. Takes top scores per cell type,... [https://rdrr.io/github/dviraran/SingleR/man/SingleR.FineTune.html]
• 25. Single Cell RNA-Seq Cell Type Prediction with SingleR [https://docs.omicsbox.biobam.com/3.3/Single-Cell-RNA-Seq-Cell-Type-Prediction/]
• 26. Top 10 Bioinformatics Tools for scRNA-seq in 2025 [https://neovarsity.org/blogs/top-bioinformatics-tools-for-scrna-seq]
• 27. Analysis, visualization, and integration of spatial datasets ... [https://satijalab.org/seurat/articles/spatial_vignette.html]
• 28. Data Integration - GitHub Pages [https://nbisweden.github.io/workshop-scRNAseq/labs/seurat/seurat_03_integration.html]
• 29. conversion SingleCellExperiment to Seurat (as. ... [https://github.com/satijalab/seurat/issues/7101]
• 30. 10X Genomics Universal 3' /5' Gene Expression (GEM-X or ... [https://ngisweden.scilifelab.se/methods/10x-genomics-3-or-5-gene-expression-gem-x-or-next-gem/]
• 31. Reference data-based cell type annotation using SingleR [https://bio-protocol.org/exchange/minidetail?id=16560873&type=30]
• 32. celldex [https://pypi.org/project/celldex/]
• 33. SingleR-inc/celldex: Collection of cell type reference ... [https://github.com/SingleR-inc/celldex]
• 34. 11. Annotation [https://www.sc-best-practices.org/cellular_structure/annotation.html]
• 35. Benchmarking cell type annotation methods for 10x Xenium ... [https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-025-06044-0]
• 36. Single Cell RNA-Seq Cell Type Prediction with SingleR [https://docs.omicsbox.biobam.com/latest/Single-Cell-RNA-Seq-Cell-Type-Prediction-SingleR/]
• 37. singler [https://pypi.org/project/singler/]
• 38. Evaluation of single-cell classifiers for single-cell RNA ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC7947964/]
• 39. Chapter 7 Advanced options | Assigning cell types with ... [https://bioconductor.org/books/release/SingleRBook/advanced-options.html]
• 40. ScaleSC: Supercharging Single Cell RNA-seq Analysis ... [https://bioinforx.com/web/blog.php?id=1000095]
• 41. GPU for Deep Learning: Critical Specs and Top 7 GPUs in ... [https://www.atlantic.net/gpu-server-hosting/gpu-for-deep-learning-critical-specs-and-top-7-gpus-in-2025/]
• 42. Choosing GPU Infrastructure for LLM Training in 2025 [https://www.whitefiber.com/blog/choosing-gpu-infrastructure]
• 43. What is Approximate Nearest Neighbor (ANN) search? [https://www.singlestore.com/blog/beginner-s-guide-to-approximate-nearest-neighbor-ann-search/]
• 44. Approximate Nearest Neighbors Oh Yeah (Annoy) [https://zilliz.com/learn/approximate-nearest-neighbor-oh-yeah-ANNOY]
• 45. scExtract: leveraging large language models for fully ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC12178070/]
• 46. Cell type annotation - CSAMA 2025 Biological Data Science [https://csama2025.bioconductor.eu/lab/5_single-cell-rnaseq/cell_type_annotation.html]
• 47. Chapter 3 Using single-cell references [https://bioconductor.org/books/release/SingleRBook/sc-mode.html]
• 48. Choosing wisely: A behind-the-scenes look at how we ... [https://www.ccdatalab.org/blog/a-behind-the-scenes-look-at-how-we-selected-cell-type-annotation-platforms-for-the-scpca-portal]
• 49. Chapter 7 Advanced options | Assigning cell types with ... [https://bioconductor.org/books/3.17/SingleRBook/advanced-options.html]
• 50. trainSingleR: Train the SingleR classifier [https://rdrr.io/bioc/SingleR/man/trainSingleR.html]
• 51. Benchmarking cell type annotation methods for 10x ... [https://pubmed.ncbi.nlm.nih.gov/39833693]
• 52. Advancing Automated Cell Type Annotation with Large ... [https://www.sciencedirect.com/science/article/pii/S2001037025004775]
• 53. Chapter 7 Cell type annotation | Basics of Single- ... [https://bioconductor.org/books/3.13/OSCA.basic/cell-type-annotation.html]
• 54. Evaluation of Cell Type Annotation R Packages on Single ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC8602772/]
• 55. The current landscape and emerging challenges of ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC12495992/]
• 56. Evaluation of Cell Type Annotation R Packages on Single- ... [https://www.sciencedirect.com/science/article/pii/S1672022920301443]
• 57. Assessing GPT-4 for cell type annotation in single-cell RNA ... [https://www.nature.com/articles/s41592-024-02235-4]
• 58. SCORE: Systematic COnsistency and Robustness ... [https://arxiv.org/html/2503.00137v1]
• 59. Benchmarking computational doublet-detection methods for ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC7897250/]
• 60. Benchmarking single-cell RNA-sequencing protocols for cell ... [https://pubmed.ncbi.nlm.nih.gov/32518403/]
• 61. A benchmark study of simulation methods for single-cell ... [https://www.nature.com/articles/s41467-021-27130-w]
• 62. SingleR [https://aran-lab.com/software/singler/]
• 63. [2409.15678] Objectively Evaluating the Reliability of Cell ... [https://arxiv.org/abs/2409.15678]
• 64. Comparative benchmarking of single-cell clustering ... [https://genomebiology.biomedcentral.com/articles/10.1186/s13059-025-03719-y]
• 65. Glowworm-cell/LICT: To efficiently transfer single-cell DEG ... [https://github.com/Glowworm-cell/LICT]
• 66. Annotation of cell types (ACT): a convenient web server for ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC10623726/]
• 67. Performance comparison of high throughput single-cell ... [https://pubmed.ncbi.nlm.nih.gov/39296129]
• 68. Performance comparison of high throughput single-cell ... [https://www.sciencedirect.com/science/article/pii/S2405844024132161]
• 69. Chapter 7 Cell type annotation | Basics of Single ... [https://bioconductor.org/books/3.14/OSCA.basic/cell-type-annotation.html]
• 70. A Review of Single-Cell RNA-Seq Annotation, Integration, and ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC10417635/]