Skills › Knowledge Work Plugins › scvi-tools › citeseq_totalvi.md
citeseq_totalvi.md
420 lines · bio-research/skills/scvi-tools/references/citeseq_totalvi.md at 2574e29
1# CITE-seq Analysis with totalVI
3This reference covers multi-modal analysis of CITE-seq data (RNA + surface proteins) using totalVI.
CITE-seq Analysis with totalVI
This reference covers multi-modal analysis of CITE-seq data (RNA + surface proteins) using totalVI.
Raw lines 1 to 45## Overview
7CITE-seq combines:
8- scRNA-seq (transcriptome)
9- Protein surface markers (antibody-derived tags, ADT)
11totalVI jointly models both modalities to:
12- Integrate across batches
13- Denoise protein signal
14- Learn joint latent representation
15- Enable cross-modal imputation
Overview
CITE-seq combines:
- scRNA-seq (transcriptome)
- Protein surface markers (antibody-derived tags, ADT)
totalVI jointly models both modalities to:
- Integrate across batches
- Denoise protein signal
- Learn joint latent representation
- Enable cross-modal imputation
17## Prerequisites
19```python
20import scvi
21import scanpy as sc
22import mudata as md
23import numpy as np
24import pandas as pd
26print(f"scvi-tools version: {scvi.__version__}")
27```
Prerequisites
import scvi
import scanpy as sc
import mudata as md
import numpy as np
import pandas as pd
print(f"scvi-tools version: {scvi.__version__}")Raw lines 17 to 28Step 1: Load CITE-seq Data
Raw lines 29 to 3031### From 10x Genomics (Cell Ranger)
33```python
34# 10x outputs separate gene expression and feature barcoding
35adata_rna = sc.read_10x_h5("filtered_feature_bc_matrix.h5", gex_only=False)
37# Separate RNA and protein
38adata_protein = adata_rna[:, adata_rna.var['feature_types'] == 'Antibody Capture'].copy()
39adata_rna = adata_rna[:, adata_rna.var['feature_types'] == 'Gene Expression'].copy()
41print(f"RNA: {adata_rna.shape}")
42print(f"Protein: {adata_protein.shape}")
43```
From 10x Genomics (Cell Ranger)
# 10x outputs separate gene expression and feature barcoding
adata_rna = sc.read_10x_h5("filtered_feature_bc_matrix.h5", gex_only=False)
# Separate RNA and protein
adata_protein = adata_rna[:, adata_rna.var['feature_types'] == 'Antibody Capture'].copy()
adata_rna = adata_rna[:, adata_rna.var['feature_types'] == 'Gene Expression'].copy()
print(f"RNA: {adata_rna.shape}")
print(f"Protein: {adata_protein.shape}")Raw lines 31 to 4445### From MuData
47```python
48# If data is in MuData format
49mdata = md.read_h5mu("cite_seq.h5mu")
51adata_rna = mdata['rna'].copy()
52adata_protein = mdata['protein'].copy()
53```
From MuData
# If data is in MuData format
mdata = md.read_h5mu("cite_seq.h5mu")
adata_rna = mdata['rna'].copy()
adata_protein = mdata['protein'].copy()Raw lines 45 to 5455### Combine into Single AnnData
57```python
58# totalVI expects protein data in obsm
59adata = adata_rna.copy()
61# Add protein expression to obsm
62adata.obsm["protein_expression"] = adata_protein.X.toarray() if hasattr(adata_protein.X, 'toarray') else adata_protein.X
64# Store protein names
65adata.uns["protein_names"] = list(adata_protein.var_names)
66```
Combine into Single AnnData
# totalVI expects protein data in obsm
adata = adata_rna.copy()
# Add protein expression to obsm
adata.obsm["protein_expression"] = adata_protein.X.toarray() if hasattr(adata_protein.X, 'toarray') else adata_protein.X
# Store protein names
adata.uns["protein_names"] = list(adata_protein.var_names)Raw lines 55 to 67Step 2: Quality Control
Raw lines 68 to 6970### RNA QC
72```python
73# Standard RNA QC
74# Handle both human (MT-) and mouse (mt-, Mt-) mitochondrial genes
75 adata.var['mt'] = (
76 adata.var_names.str.startswith('MT-') |
77 adata.var_names.str.startswith('mt-') |
78 adata.var_names.str.startswith('Mt-')
79 )
80sc.pp.calculate_qc_metrics(adata, qc_vars=['mt'], inplace=True)
82# Filter cells
83adata = adata[adata.obs['n_genes_by_counts'] > 200].copy()
84adata = adata[adata.obs['pct_counts_mt'] < 20].copy()
86# Filter genes
87sc.pp.filter_genes(adata, min_cells=3)
88```
RNA QC
# Standard RNA QC
# Handle both human (MT-) and mouse (mt-, Mt-) mitochondrial genes
adata.var['mt'] = (
adata.var_names.str.startswith('MT-') |
adata.var_names.str.startswith('mt-') |
adata.var_names.str.startswith('Mt-')
)
sc.pp.calculate_qc_metrics(adata, qc_vars=['mt'], inplace=True)
# Filter cells
adata = adata[adata.obs['n_genes_by_counts'] > 200].copy()
adata = adata[adata.obs['pct_counts_mt'] < 20].copy()
# Filter genes
sc.pp.filter_genes(adata, min_cells=3)Raw lines 70 to 8990### Protein QC
92```python
93# Protein QC
94protein_counts = adata.obsm["protein_expression"]
95print(f"Protein counts per cell: min={protein_counts.sum(1).min():.0f}, max={protein_counts.sum(1).max():.0f}")
97# Check for isotype controls
98# Isotype controls should have low counts
99protein_names = adata.uns["protein_names"]
100for i, name in enumerate(protein_names):
101 if 'isotype' in name.lower() or 'control' in name.lower():
102 print(f"{name}: mean={protein_counts[:, i].mean():.1f}")
103```
Protein QC
# Protein QC
protein_counts = adata.obsm["protein_expression"]
print(f"Protein counts per cell: min={protein_counts.sum(1).min():.0f}, max={protein_counts.sum(1).max():.0f}")
# Check for isotype controls
# Isotype controls should have low counts
protein_names = adata.uns["protein_names"]
for i, name in enumerate(protein_names):
if 'isotype' in name.lower() or 'control' in name.lower():
print(f"{name}: mean={protein_counts[:, i].mean():.1f}")Raw lines 90 to 104Step 3: Data Preparation
Raw lines 105 to 106107### Store Raw Counts
109```python
110# Store RNA counts
111adata.layers["counts"] = adata.X.copy()
113# Protein must be raw ADT counts (NOT CLR-normalized)
114# WARNING: If importing from Seurat, ensure you use raw counts, not CLR-normalized data
115# Seurat's NormalizeData(normalization.method = "CLR") transforms counts - use the original assay
116```
Store Raw Counts
# Store RNA counts
adata.layers["counts"] = adata.X.copy()
# Protein must be raw ADT counts (NOT CLR-normalized)
# WARNING: If importing from Seurat, ensure you use raw counts, not CLR-normalized data
# Seurat's NormalizeData(normalization.method = "CLR") transforms counts - use the original assayRaw lines 107 to 117118### HVG Selection for RNA
120```python
121# Select HVGs for RNA
122# Note: totalVI uses all proteins regardless of HVG
124sc.pp.highly_variable_genes(
125 adata,
126 n_top_genes=4000, # Use more for CITE-seq
127 flavor="seurat_v3",
128 batch_key="batch" if "batch" in adata.obs else None,
129 layer="counts"
130)
132# Subset to HVGs
133adata = adata[:, adata.var["highly_variable"]].copy()
134```
HVG Selection for RNA
# Select HVGs for RNA
# Note: totalVI uses all proteins regardless of HVG
sc.pp.highly_variable_genes(
adata,
n_top_genes=4000, # Use more for CITE-seq
flavor="seurat_v3",
batch_key="batch" if "batch" in adata.obs else None,
layer="counts"
)
# Subset to HVGs
adata = adata[:, adata.var["highly_variable"]].copy()Raw lines 118 to 135136## Step 4: Setup and Train totalVI
138```python
139# Setup AnnData for totalVI
140scvi.model.TOTALVI.setup_anndata(
141 adata,
142 layer="counts",
143 protein_expression_obsm_key="protein_expression",
144 batch_key="batch" # Optional
145)
147# Create model
148model = scvi.model.TOTALVI(
149 adata,
150 n_latent=20,
151 latent_distribution="normal" # or "ln" for log-normal
152)
154# Train
155model.train(
156 max_epochs=200,
157 early_stopping=True,
158 batch_size=128
159)
161# Check training
162model.history['elbo_train'].plot()
163```
Step 4: Setup and Train totalVI
# Setup AnnData for totalVI
scvi.model.TOTALVI.setup_anndata(
adata,
layer="counts",
protein_expression_obsm_key="protein_expression",
batch_key="batch" # Optional
)
# Create model
model = scvi.model.TOTALVI(
adata,
n_latent=20,
latent_distribution="normal" # or "ln" for log-normal
)
# Train
model.train(
max_epochs=200,
early_stopping=True,
batch_size=128
)
# Check training
model.history['elbo_train'].plot()Raw lines 136 to 164165## Step 5: Get Latent Representation
167```python
168# Joint latent space
169adata.obsm["X_totalVI"] = model.get_latent_representation()
171# Clustering and visualization
172sc.pp.neighbors(adata, use_rep="X_totalVI")
173sc.tl.umap(adata)
174sc.tl.leiden(adata, resolution=1.0)
176sc.pl.umap(adata, color=['leiden', 'batch'])
177```
Step 5: Get Latent Representation
# Joint latent space
adata.obsm["X_totalVI"] = model.get_latent_representation()
# Clustering and visualization
sc.pp.neighbors(adata, use_rep="X_totalVI")
sc.tl.umap(adata)
sc.tl.leiden(adata, resolution=1.0)
sc.pl.umap(adata, color=['leiden', 'batch'])Raw lines 165 to 178179## Step 6: Denoised Protein Expression
181```python
182# Get denoised protein values
183# This removes background noise from protein measurements
185_, protein_denoised = model.get_normalized_expression(
186 return_mean=True,
187 transform_batch="batch1" # Optional: normalize to specific batch
188)
190# Add to adata
191adata.obsm["protein_denoised"] = protein_denoised
193# Visualize denoised proteins
194protein_names = adata.uns["protein_names"]
195for i, protein in enumerate(protein_names[:5]):
196 adata.obs[f"denoised_{protein}"] = protein_denoised[:, i]
198sc.pl.umap(adata, color=[f"denoised_{p}" for p in protein_names[:5]])
199```
Step 6: Denoised Protein Expression
# Get denoised protein values
# This removes background noise from protein measurements
_, protein_denoised = model.get_normalized_expression(
return_mean=True,
transform_batch="batch1" # Optional: normalize to specific batch
)
# Add to adata
adata.obsm["protein_denoised"] = protein_denoised
# Visualize denoised proteins
protein_names = adata.uns["protein_names"]
for i, protein in enumerate(protein_names[:5]):
adata.obs[f"denoised_{protein}"] = protein_denoised[:, i]
sc.pl.umap(adata, color=[f"denoised_{p}" for p in protein_names[:5]])Raw lines 179 to 200201## Step 7: Normalized RNA Expression
203```python
204# Get normalized RNA expression
205rna_normalized, _ = model.get_normalized_expression(
206 return_mean=True
207)
209# Store
210adata.layers["totalVI_normalized"] = rna_normalized
211```
Step 7: Normalized RNA Expression
# Get normalized RNA expression
rna_normalized, _ = model.get_normalized_expression(
return_mean=True
)
# Store
adata.layers["totalVI_normalized"] = rna_normalizedRaw lines 201 to 212Step 8: Differential Expression
Raw lines 213 to 214215### RNA Differential Expression
217```python
218# DE between clusters
219de_rna = model.differential_expression(
220 groupby="leiden",
221 group1="0",
222 group2="1"
223)
225# Filter significant genes
226de_sig = de_rna[
227 (de_rna['is_de_fdr_0.05']) &
228 (abs(de_rna['lfc_mean']) > 1)
229]
231print(f"Significant DE genes: {len(de_sig)}")
232```
RNA Differential Expression
# DE between clusters
de_rna = model.differential_expression(
groupby="leiden",
group1="0",
group2="1"
)
# Filter significant genes
de_sig = de_rna[
(de_rna['is_de_fdr_0.05']) &
(abs(de_rna['lfc_mean']) > 1)
]
print(f"Significant DE genes: {len(de_sig)}")Raw lines 215 to 233234### Protein Differential Expression
236```python
237# Protein DE
238de_protein = model.differential_expression(
239 groupby="leiden",
240 group1="0",
241 group2="1",
242 mode="protein"
243)
245print(de_protein.head(20))
246```
Protein Differential Expression
# Protein DE
de_protein = model.differential_expression(
groupby="leiden",
group1="0",
group2="1",
mode="protein"
)
print(de_protein.head(20))Raw lines 234 to 247Step 9: Visualization
Raw lines 248 to 249250### Protein Expression on UMAP
252```python
253# Denoised protein on UMAP
254import matplotlib.pyplot as plt
256proteins_to_plot = ["CD3", "CD4", "CD8", "CD19", "CD14"]
258fig, axes = plt.subplots(1, len(proteins_to_plot), figsize=(4*len(proteins_to_plot), 4))
259for ax, protein in zip(axes, proteins_to_plot):
260 idx = adata.uns["protein_names"].index(protein)
261 sc.pl.umap(
262 adata,
263 color=adata.obsm["protein_denoised"][:, idx],
264 ax=ax,
265 title=protein,
266 show=False
267 )
268plt.tight_layout()
269```
Protein Expression on UMAP
# Denoised protein on UMAP
import matplotlib.pyplot as plt
proteins_to_plot = ["CD3", "CD4", "CD8", "CD19", "CD14"]
fig, axes = plt.subplots(1, len(proteins_to_plot), figsize=(4*len(proteins_to_plot), 4))
for ax, protein in zip(axes, proteins_to_plot):
idx = adata.uns["protein_names"].index(protein)
sc.pl.umap(
adata,
color=adata.obsm["protein_denoised"][:, idx],
ax=ax,
title=protein,
show=False
)
plt.tight_layout()Raw lines 250 to 270271### Joint Heatmap
273```python
274# Heatmap of top genes and proteins per cluster
275sc.pl.dotplot(
276 adata,
277 var_names=de_sig.index[:20].tolist(),
278 groupby="leiden",
279 layer="totalVI_normalized"
280)
281```
Joint Heatmap
# Heatmap of top genes and proteins per cluster
sc.pl.dotplot(
adata,
var_names=de_sig.index[:20].tolist(),
groupby="leiden",
layer="totalVI_normalized"
)Raw lines 271 to 282283## Step 10: Cell Type Annotation
285```python
286# Use both RNA and protein markers for annotation
288# RNA markers
289rna_markers = {
290 'T cells': ['CD3D', 'CD3E'],
291 'CD4 T': ['CD4'],
292 'CD8 T': ['CD8A', 'CD8B'],
293 'B cells': ['CD19', 'MS4A1'],
294 'Monocytes': ['CD14', 'LYZ']
295}
297# Check denoised protein expression
298for i, protein in enumerate(adata.uns["protein_names"]):
299 if any(m in protein for m in ['CD3', 'CD4', 'CD8', 'CD19', 'CD14']):
300 print(f"{protein}: cluster means")
301 for cluster in adata.obs['leiden'].unique():
302 mask = adata.obs['leiden'] == cluster
303 mean_expr = adata.obsm["protein_denoised"][mask, i].mean()
304 print(f" Cluster {cluster}: {mean_expr:.2f}")
305```
Step 10: Cell Type Annotation
# Use both RNA and protein markers for annotation
# RNA markers
rna_markers = {
'T cells': ['CD3D', 'CD3E'],
'CD4 T': ['CD4'],
'CD8 T': ['CD8A', 'CD8B'],
'B cells': ['CD19', 'MS4A1'],
'Monocytes': ['CD14', 'LYZ']
}
# Check denoised protein expression
for i, protein in enumerate(adata.uns["protein_names"]):
if any(m in protein for m in ['CD3', 'CD4', 'CD8', 'CD19', 'CD14']):
print(f"{protein}: cluster means")
for cluster in adata.obs['leiden'].unique():
mask = adata.obs['leiden'] == cluster
mean_expr = adata.obsm["protein_denoised"][mask, i].mean()
print(f" Cluster {cluster}: {mean_expr:.2f}")Raw lines 283 to 306307## Complete Pipeline
309```python
310def analyze_citeseq(
311 adata_rna,
312 adata_protein,
313 batch_key=None,
314 n_top_genes=4000,
315 n_latent=20
316):
317 """
318 Complete CITE-seq analysis with totalVI.
320 Parameters
321 ----------
322 adata_rna : AnnData
323 RNA expression (raw counts)
324 adata_protein : AnnData
325 Protein expression (raw counts)
326 batch_key : str, optional
327 Batch column in obs
328 n_top_genes : int
329 Number of HVGs
330 n_latent : int
331 Latent dimensions
333 Returns
334 -------
335 Tuple of (processed AnnData, trained model)
336 """
337 import scvi
338 import scanpy as sc
340 # Ensure same cells
341 common_cells = adata_rna.obs_names.intersection(adata_protein.obs_names)
342 adata = adata_rna[common_cells].copy()
343 adata_protein = adata_protein[common_cells].copy()
345 # Add protein to obsm
346 adata.obsm["protein_expression"] = adata_protein.X.toarray() if hasattr(adata_protein.X, 'toarray') else adata_protein.X
347 adata.uns["protein_names"] = list(adata_protein.var_names)
349 # RNA QC
350 # Handle both human (MT-) and mouse (mt-, Mt-) mitochondrial genes
351 adata.var['mt'] = (
352 adata.var_names.str.startswith('MT-') |
353 adata.var_names.str.startswith('mt-') |
354 adata.var_names.str.startswith('Mt-')
355 )
356 sc.pp.calculate_qc_metrics(adata, qc_vars=['mt'], inplace=True)
357 adata = adata[adata.obs['pct_counts_mt'] < 20].copy()
358 sc.pp.filter_genes(adata, min_cells=3)
360 # Store counts
361 adata.layers["counts"] = adata.X.copy()
363 # HVG selection
364 sc.pp.highly_variable_genes(
365 adata,
366 n_top_genes=n_top_genes,
367 flavor="seurat_v3",
368 batch_key=batch_key,
369 layer="counts"
370 )
371 adata = adata[:, adata.var["highly_variable"]].copy()
373 # Setup totalVI
374 scvi.model.TOTALVI.setup_anndata(
375 adata,
376 layer="counts",
377 protein_expression_obsm_key="protein_expression",
378 batch_key=batch_key
379 )
381 # Train
382 model = scvi.model.TOTALVI(adata, n_latent=n_latent)
383 model.train(max_epochs=200, early_stopping=True)
385 # Get representations
386 adata.obsm["X_totalVI"] = model.get_latent_representation()
387 rna_norm, protein_denoised = model.get_normalized_expression(return_mean=True)
388 adata.layers["totalVI_normalized"] = rna_norm
389 adata.obsm["protein_denoised"] = protein_denoised
391 # Clustering
392 sc.pp.neighbors(adata, use_rep="X_totalVI")
393 sc.tl.umap(adata)
394 sc.tl.leiden(adata)
396 return adata, model
398# Usage
399adata, model = analyze_citeseq(
400 adata_rna,
401 adata_protein,
402 batch_key="batch"
403)
405# Visualize
406sc.pl.umap(adata, color=['leiden', 'batch'])
407```
Complete Pipeline
def analyze_citeseq(
adata_rna,
adata_protein,
batch_key=None,
n_top_genes=4000,
n_latent=20
):
"""
Complete CITE-seq analysis with totalVI.
Parameters
----------
adata_rna : AnnData
RNA expression (raw counts)
adata_protein : AnnData
Protein expression (raw counts)
batch_key : str, optional
Batch column in obs
n_top_genes : int
Number of HVGs
n_latent : int
Latent dimensions
Returns
-------
Tuple of (processed AnnData, trained model)
"""
import scvi
import scanpy as sc
# Ensure same cells
common_cells = adata_rna.obs_names.intersection(adata_protein.obs_names)
adata = adata_rna[common_cells].copy()
adata_protein = adata_protein[common_cells].copy()
# Add protein to obsm
adata.obsm["protein_expression"] = adata_protein.X.toarray() if hasattr(adata_protein.X, 'toarray') else adata_protein.X
adata.uns["protein_names"] = list(adata_protein.var_names)
# RNA QC
# Handle both human (MT-) and mouse (mt-, Mt-) mitochondrial genes
adata.var['mt'] = (
adata.var_names.str.startswith('MT-') |
adata.var_names.str.startswith('mt-') |
adata.var_names.str.startswith('Mt-')
)
sc.pp.calculate_qc_metrics(adata, qc_vars=['mt'], inplace=True)
adata = adata[adata.obs['pct_counts_mt'] < 20].copy()
sc.pp.filter_genes(adata, min_cells=3)
# Store counts
adata.layers["counts"] = adata.X.copy()
# HVG selection
sc.pp.highly_variable_genes(
adata,
n_top_genes=n_top_genes,
flavor="seurat_v3",
batch_key=batch_key,
layer="counts"
)
adata = adata[:, adata.var["highly_variable"]].copy()
# Setup totalVI
scvi.model.TOTALVI.setup_anndata(
adata,
layer="counts",
protein_expression_obsm_key="protein_expression",
batch_key=batch_key
)
# Train
model = scvi.model.TOTALVI(adata, n_latent=n_latent)
model.train(max_epochs=200, early_stopping=True)
# Get representations
adata.obsm["X_totalVI"] = model.get_latent_representation()
rna_norm, protein_denoised = model.get_normalized_expression(return_mean=True)
adata.layers["totalVI_normalized"] = rna_norm
adata.obsm["protein_denoised"] = protein_denoised
# Clustering
sc.pp.neighbors(adata, use_rep="X_totalVI")
sc.tl.umap(adata)
sc.tl.leiden(adata)
return adata, model
# Usage
adata, model = analyze_citeseq(
adata_rna,
adata_protein,
batch_key="batch"
)
# Visualize
sc.pl.umap(adata, color=['leiden', 'batch'])Raw lines 307 to 408409## Troubleshooting
411| Issue | Cause | Solution |
412|-------|-------|----------|
413| Protein signal noisy | Background not removed | Use get_normalized_expression with denoising |
414| Batch effects persist | Need batch_key | Ensure batch_key is specified |
415| Memory error | Too many genes | Reduce n_top_genes |
416| Poor protein clustering | Few proteins | Normal - totalVI uses RNA for structure |
Troubleshooting
| Issue | Cause | Solution |
|---|---|---|
| Protein signal noisy | Background not removed | Use get_normalized_expression with denoising |
| Batch effects persist | Need batch_key | Ensure batch_key is specified |
| Memory error | Too many genes | Reduce n_top_genes |
| Poor protein clustering | Few proteins | Normal - totalVI uses RNA for structure |
418## Key References
420- Gayoso et al. (2021) "Joint probabilistic modeling of single-cell multi-omic data with totalVI"
Key References
- Gayoso et al. (2021) "Joint probabilistic modeling of single-cell multi-omic data with totalVI"