Skill Tracer
Skills › Knowledge Work Plugins › scvi-tools › citeseq_totalvi.md

citeseq_totalvi.md

420 lines · bio-research/skills/scvi-tools/references/citeseq_totalvi.md at 2574e29

Open on GitHub
1# CITE-seq Analysis with totalVI
2
3This reference covers multi-modal analysis of CITE-seq data (RNA + surface proteins) using totalVI.
4

CITE-seq Analysis with totalVI

This reference covers multi-modal analysis of CITE-seq data (RNA + surface proteins) using totalVI.

Raw lines 1 to 4
5## Overview
6
7CITE-seq combines:
8- scRNA-seq (transcriptome)
9- Protein surface markers (antibody-derived tags, ADT)
11totalVI jointly models both modalities to:
12- Integrate across batches
13- Denoise protein signal
14- Learn joint latent representation
15- Enable cross-modal imputation

Overview

CITE-seq combines:

  • scRNA-seq (transcriptome)
  • Protein surface markers (antibody-derived tags, ADT)

totalVI jointly models both modalities to:

  • Integrate across batches
  • Denoise protein signal
  • Learn joint latent representation
  • Enable cross-modal imputation
Raw lines 5 to 16
17## Prerequisites
19```python
20import scvi
21import scanpy as sc
22import mudata as md
23import numpy as np
24import pandas as pd
26print(f"scvi-tools version: {scvi.__version__}")
27```

Prerequisites

import scvi
import scanpy as sc
import mudata as md
import numpy as np
import pandas as pd

print(f"scvi-tools version: {scvi.__version__}")
Raw lines 17 to 28
29## Step 1: Load CITE-seq Data

Step 1: Load CITE-seq Data

Raw lines 29 to 30
31### From 10x Genomics (Cell Ranger)
33```python
34# 10x outputs separate gene expression and feature barcoding
35adata_rna = sc.read_10x_h5("filtered_feature_bc_matrix.h5", gex_only=False)
37# Separate RNA and protein
38adata_protein = adata_rna[:, adata_rna.var['feature_types'] == 'Antibody Capture'].copy()
39adata_rna = adata_rna[:, adata_rna.var['feature_types'] == 'Gene Expression'].copy()
41print(f"RNA: {adata_rna.shape}")
42print(f"Protein: {adata_protein.shape}")
43```

From 10x Genomics (Cell Ranger)

# 10x outputs separate gene expression and feature barcoding
adata_rna = sc.read_10x_h5("filtered_feature_bc_matrix.h5", gex_only=False)

# Separate RNA and protein
adata_protein = adata_rna[:, adata_rna.var['feature_types'] == 'Antibody Capture'].copy()
adata_rna = adata_rna[:, adata_rna.var['feature_types'] == 'Gene Expression'].copy()

print(f"RNA: {adata_rna.shape}")
print(f"Protein: {adata_protein.shape}")
Raw lines 31 to 44
45### From MuData
47```python
48# If data is in MuData format
49mdata = md.read_h5mu("cite_seq.h5mu")
51adata_rna = mdata['rna'].copy()
52adata_protein = mdata['protein'].copy()
53```

From MuData

# If data is in MuData format
mdata = md.read_h5mu("cite_seq.h5mu")

adata_rna = mdata['rna'].copy()
adata_protein = mdata['protein'].copy()
Raw lines 45 to 54
55### Combine into Single AnnData
57```python
58# totalVI expects protein data in obsm
59adata = adata_rna.copy()
61# Add protein expression to obsm
62adata.obsm["protein_expression"] = adata_protein.X.toarray() if hasattr(adata_protein.X, 'toarray') else adata_protein.X
64# Store protein names
65adata.uns["protein_names"] = list(adata_protein.var_names)
66```

Combine into Single AnnData

# totalVI expects protein data in obsm
adata = adata_rna.copy()

# Add protein expression to obsm
adata.obsm["protein_expression"] = adata_protein.X.toarray() if hasattr(adata_protein.X, 'toarray') else adata_protein.X

# Store protein names
adata.uns["protein_names"] = list(adata_protein.var_names)
Raw lines 55 to 67
68## Step 2: Quality Control

Step 2: Quality Control

Raw lines 68 to 69
70### RNA QC
72```python
73# Standard RNA QC
74# Handle both human (MT-) and mouse (mt-, Mt-) mitochondrial genes
75 adata.var['mt'] = (
76 adata.var_names.str.startswith('MT-') |
77 adata.var_names.str.startswith('mt-') |
78 adata.var_names.str.startswith('Mt-')
79 )
80sc.pp.calculate_qc_metrics(adata, qc_vars=['mt'], inplace=True)
82# Filter cells
83adata = adata[adata.obs['n_genes_by_counts'] > 200].copy()
84adata = adata[adata.obs['pct_counts_mt'] < 20].copy()
86# Filter genes
87sc.pp.filter_genes(adata, min_cells=3)
88```

RNA QC

# Standard RNA QC
# Handle both human (MT-) and mouse (mt-, Mt-) mitochondrial genes
    adata.var['mt'] = (
        adata.var_names.str.startswith('MT-') |
        adata.var_names.str.startswith('mt-') |
        adata.var_names.str.startswith('Mt-')
    )
sc.pp.calculate_qc_metrics(adata, qc_vars=['mt'], inplace=True)

# Filter cells
adata = adata[adata.obs['n_genes_by_counts'] > 200].copy()
adata = adata[adata.obs['pct_counts_mt'] < 20].copy()

# Filter genes
sc.pp.filter_genes(adata, min_cells=3)
Raw lines 70 to 89
90### Protein QC
92```python
93# Protein QC
94protein_counts = adata.obsm["protein_expression"]
95print(f"Protein counts per cell: min={protein_counts.sum(1).min():.0f}, max={protein_counts.sum(1).max():.0f}")
97# Check for isotype controls
98# Isotype controls should have low counts
99protein_names = adata.uns["protein_names"]
100for i, name in enumerate(protein_names):
101 if 'isotype' in name.lower() or 'control' in name.lower():
102 print(f"{name}: mean={protein_counts[:, i].mean():.1f}")
103```

Protein QC

# Protein QC
protein_counts = adata.obsm["protein_expression"]
print(f"Protein counts per cell: min={protein_counts.sum(1).min():.0f}, max={protein_counts.sum(1).max():.0f}")

# Check for isotype controls
# Isotype controls should have low counts
protein_names = adata.uns["protein_names"]
for i, name in enumerate(protein_names):
    if 'isotype' in name.lower() or 'control' in name.lower():
        print(f"{name}: mean={protein_counts[:, i].mean():.1f}")
Raw lines 90 to 104
105## Step 3: Data Preparation

Step 3: Data Preparation

Raw lines 105 to 106
107### Store Raw Counts
109```python
110# Store RNA counts
111adata.layers["counts"] = adata.X.copy()
113# Protein must be raw ADT counts (NOT CLR-normalized)
114# WARNING: If importing from Seurat, ensure you use raw counts, not CLR-normalized data
115# Seurat's NormalizeData(normalization.method = "CLR") transforms counts - use the original assay
116```

Store Raw Counts

# Store RNA counts
adata.layers["counts"] = adata.X.copy()

# Protein must be raw ADT counts (NOT CLR-normalized)
# WARNING: If importing from Seurat, ensure you use raw counts, not CLR-normalized data
# Seurat's NormalizeData(normalization.method = "CLR") transforms counts - use the original assay
Raw lines 107 to 117
118### HVG Selection for RNA
120```python
121# Select HVGs for RNA
122# Note: totalVI uses all proteins regardless of HVG
124sc.pp.highly_variable_genes(
125 adata,
126 n_top_genes=4000, # Use more for CITE-seq
127 flavor="seurat_v3",
128 batch_key="batch" if "batch" in adata.obs else None,
129 layer="counts"
132# Subset to HVGs
133adata = adata[:, adata.var["highly_variable"]].copy()
134```

HVG Selection for RNA

# Select HVGs for RNA
# Note: totalVI uses all proteins regardless of HVG

sc.pp.highly_variable_genes(
    adata,
    n_top_genes=4000,  # Use more for CITE-seq
    flavor="seurat_v3",
    batch_key="batch" if "batch" in adata.obs else None,
    layer="counts"
)

# Subset to HVGs
adata = adata[:, adata.var["highly_variable"]].copy()
Raw lines 118 to 135
136## Step 4: Setup and Train totalVI
138```python
139# Setup AnnData for totalVI
140scvi.model.TOTALVI.setup_anndata(
141 adata,
142 layer="counts",
143 protein_expression_obsm_key="protein_expression",
144 batch_key="batch" # Optional
147# Create model
148model = scvi.model.TOTALVI(
149 adata,
150 n_latent=20,
151 latent_distribution="normal" # or "ln" for log-normal
154# Train
155model.train(
156 max_epochs=200,
157 early_stopping=True,
158 batch_size=128
161# Check training
162model.history['elbo_train'].plot()
163```

Step 4: Setup and Train totalVI

# Setup AnnData for totalVI
scvi.model.TOTALVI.setup_anndata(
    adata,
    layer="counts",
    protein_expression_obsm_key="protein_expression",
    batch_key="batch"  # Optional
)

# Create model
model = scvi.model.TOTALVI(
    adata,
    n_latent=20,
    latent_distribution="normal"  # or "ln" for log-normal
)

# Train
model.train(
    max_epochs=200,
    early_stopping=True,
    batch_size=128
)

# Check training
model.history['elbo_train'].plot()
Raw lines 136 to 164
165## Step 5: Get Latent Representation
167```python
168# Joint latent space
169adata.obsm["X_totalVI"] = model.get_latent_representation()
171# Clustering and visualization
172sc.pp.neighbors(adata, use_rep="X_totalVI")
173sc.tl.umap(adata)
174sc.tl.leiden(adata, resolution=1.0)
176sc.pl.umap(adata, color=['leiden', 'batch'])
177```

Step 5: Get Latent Representation

# Joint latent space
adata.obsm["X_totalVI"] = model.get_latent_representation()

# Clustering and visualization
sc.pp.neighbors(adata, use_rep="X_totalVI")
sc.tl.umap(adata)
sc.tl.leiden(adata, resolution=1.0)

sc.pl.umap(adata, color=['leiden', 'batch'])
Raw lines 165 to 178
179## Step 6: Denoised Protein Expression
181```python
182# Get denoised protein values
183# This removes background noise from protein measurements
185_, protein_denoised = model.get_normalized_expression(
186 return_mean=True,
187 transform_batch="batch1" # Optional: normalize to specific batch
190# Add to adata
191adata.obsm["protein_denoised"] = protein_denoised
193# Visualize denoised proteins
194protein_names = adata.uns["protein_names"]
195for i, protein in enumerate(protein_names[:5]):
196 adata.obs[f"denoised_{protein}"] = protein_denoised[:, i]
198sc.pl.umap(adata, color=[f"denoised_{p}" for p in protein_names[:5]])
199```

Step 6: Denoised Protein Expression

# Get denoised protein values
# This removes background noise from protein measurements

_, protein_denoised = model.get_normalized_expression(
    return_mean=True,
    transform_batch="batch1"  # Optional: normalize to specific batch
)

# Add to adata
adata.obsm["protein_denoised"] = protein_denoised

# Visualize denoised proteins
protein_names = adata.uns["protein_names"]
for i, protein in enumerate(protein_names[:5]):
    adata.obs[f"denoised_{protein}"] = protein_denoised[:, i]

sc.pl.umap(adata, color=[f"denoised_{p}" for p in protein_names[:5]])
Raw lines 179 to 200
201## Step 7: Normalized RNA Expression
203```python
204# Get normalized RNA expression
205rna_normalized, _ = model.get_normalized_expression(
206 return_mean=True
209# Store
210adata.layers["totalVI_normalized"] = rna_normalized
211```

Step 7: Normalized RNA Expression

# Get normalized RNA expression
rna_normalized, _ = model.get_normalized_expression(
    return_mean=True
)

# Store
adata.layers["totalVI_normalized"] = rna_normalized
Raw lines 201 to 212
213## Step 8: Differential Expression

Step 8: Differential Expression

Raw lines 213 to 214
215### RNA Differential Expression
217```python
218# DE between clusters
219de_rna = model.differential_expression(
220 groupby="leiden",
221 group1="0",
222 group2="1"
225# Filter significant genes
226de_sig = de_rna[
227 (de_rna['is_de_fdr_0.05']) &
228 (abs(de_rna['lfc_mean']) > 1)
231print(f"Significant DE genes: {len(de_sig)}")
232```

RNA Differential Expression

# DE between clusters
de_rna = model.differential_expression(
    groupby="leiden",
    group1="0",
    group2="1"
)

# Filter significant genes
de_sig = de_rna[
    (de_rna['is_de_fdr_0.05']) &
    (abs(de_rna['lfc_mean']) > 1)
]

print(f"Significant DE genes: {len(de_sig)}")
Raw lines 215 to 233
234### Protein Differential Expression
236```python
237# Protein DE
238de_protein = model.differential_expression(
239 groupby="leiden",
240 group1="0",
241 group2="1",
242 mode="protein"
245print(de_protein.head(20))
246```

Protein Differential Expression

# Protein DE
de_protein = model.differential_expression(
    groupby="leiden",
    group1="0",
    group2="1",
    mode="protein"
)

print(de_protein.head(20))
Raw lines 234 to 247
248## Step 9: Visualization

Step 9: Visualization

Raw lines 248 to 249
250### Protein Expression on UMAP
252```python
253# Denoised protein on UMAP
254import matplotlib.pyplot as plt
256proteins_to_plot = ["CD3", "CD4", "CD8", "CD19", "CD14"]
258fig, axes = plt.subplots(1, len(proteins_to_plot), figsize=(4*len(proteins_to_plot), 4))
259for ax, protein in zip(axes, proteins_to_plot):
260 idx = adata.uns["protein_names"].index(protein)
261 sc.pl.umap(
262 adata,
263 color=adata.obsm["protein_denoised"][:, idx],
264 ax=ax,
265 title=protein,
266 show=False
267 )
268plt.tight_layout()
269```

Protein Expression on UMAP

# Denoised protein on UMAP
import matplotlib.pyplot as plt

proteins_to_plot = ["CD3", "CD4", "CD8", "CD19", "CD14"]

fig, axes = plt.subplots(1, len(proteins_to_plot), figsize=(4*len(proteins_to_plot), 4))
for ax, protein in zip(axes, proteins_to_plot):
    idx = adata.uns["protein_names"].index(protein)
    sc.pl.umap(
        adata,
        color=adata.obsm["protein_denoised"][:, idx],
        ax=ax,
        title=protein,
        show=False
    )
plt.tight_layout()
Raw lines 250 to 270
271### Joint Heatmap
273```python
274# Heatmap of top genes and proteins per cluster
275sc.pl.dotplot(
276 adata,
277 var_names=de_sig.index[:20].tolist(),
278 groupby="leiden",
279 layer="totalVI_normalized"
281```

Joint Heatmap

# Heatmap of top genes and proteins per cluster
sc.pl.dotplot(
    adata,
    var_names=de_sig.index[:20].tolist(),
    groupby="leiden",
    layer="totalVI_normalized"
)
Raw lines 271 to 282
283## Step 10: Cell Type Annotation
285```python
286# Use both RNA and protein markers for annotation
288# RNA markers
289rna_markers = {
290 'T cells': ['CD3D', 'CD3E'],
291 'CD4 T': ['CD4'],
292 'CD8 T': ['CD8A', 'CD8B'],
293 'B cells': ['CD19', 'MS4A1'],
294 'Monocytes': ['CD14', 'LYZ']
297# Check denoised protein expression
298for i, protein in enumerate(adata.uns["protein_names"]):
299 if any(m in protein for m in ['CD3', 'CD4', 'CD8', 'CD19', 'CD14']):
300 print(f"{protein}: cluster means")
301 for cluster in adata.obs['leiden'].unique():
302 mask = adata.obs['leiden'] == cluster
303 mean_expr = adata.obsm["protein_denoised"][mask, i].mean()
304 print(f" Cluster {cluster}: {mean_expr:.2f}")
305```

Step 10: Cell Type Annotation

# Use both RNA and protein markers for annotation

# RNA markers
rna_markers = {
    'T cells': ['CD3D', 'CD3E'],
    'CD4 T': ['CD4'],
    'CD8 T': ['CD8A', 'CD8B'],
    'B cells': ['CD19', 'MS4A1'],
    'Monocytes': ['CD14', 'LYZ']
}

# Check denoised protein expression
for i, protein in enumerate(adata.uns["protein_names"]):
    if any(m in protein for m in ['CD3', 'CD4', 'CD8', 'CD19', 'CD14']):
        print(f"{protein}: cluster means")
        for cluster in adata.obs['leiden'].unique():
            mask = adata.obs['leiden'] == cluster
            mean_expr = adata.obsm["protein_denoised"][mask, i].mean()
            print(f"  Cluster {cluster}: {mean_expr:.2f}")
Raw lines 283 to 306
307## Complete Pipeline
309```python
310def analyze_citeseq(
311 adata_rna,
312 adata_protein,
313 batch_key=None,
314 n_top_genes=4000,
315 n_latent=20
316):
317 """
318 Complete CITE-seq analysis with totalVI.
320 Parameters
321 ----------
322 adata_rna : AnnData
323 RNA expression (raw counts)
324 adata_protein : AnnData
325 Protein expression (raw counts)
326 batch_key : str, optional
327 Batch column in obs
328 n_top_genes : int
329 Number of HVGs
330 n_latent : int
331 Latent dimensions
333 Returns
334 -------
335 Tuple of (processed AnnData, trained model)
336 """
337 import scvi
338 import scanpy as sc
340 # Ensure same cells
341 common_cells = adata_rna.obs_names.intersection(adata_protein.obs_names)
342 adata = adata_rna[common_cells].copy()
343 adata_protein = adata_protein[common_cells].copy()
345 # Add protein to obsm
346 adata.obsm["protein_expression"] = adata_protein.X.toarray() if hasattr(adata_protein.X, 'toarray') else adata_protein.X
347 adata.uns["protein_names"] = list(adata_protein.var_names)
349 # RNA QC
350 # Handle both human (MT-) and mouse (mt-, Mt-) mitochondrial genes
351 adata.var['mt'] = (
352 adata.var_names.str.startswith('MT-') |
353 adata.var_names.str.startswith('mt-') |
354 adata.var_names.str.startswith('Mt-')
355 )
356 sc.pp.calculate_qc_metrics(adata, qc_vars=['mt'], inplace=True)
357 adata = adata[adata.obs['pct_counts_mt'] < 20].copy()
358 sc.pp.filter_genes(adata, min_cells=3)
360 # Store counts
361 adata.layers["counts"] = adata.X.copy()
363 # HVG selection
364 sc.pp.highly_variable_genes(
365 adata,
366 n_top_genes=n_top_genes,
367 flavor="seurat_v3",
368 batch_key=batch_key,
369 layer="counts"
370 )
371 adata = adata[:, adata.var["highly_variable"]].copy()
373 # Setup totalVI
374 scvi.model.TOTALVI.setup_anndata(
375 adata,
376 layer="counts",
377 protein_expression_obsm_key="protein_expression",
378 batch_key=batch_key
379 )
381 # Train
382 model = scvi.model.TOTALVI(adata, n_latent=n_latent)
383 model.train(max_epochs=200, early_stopping=True)
385 # Get representations
386 adata.obsm["X_totalVI"] = model.get_latent_representation()
387 rna_norm, protein_denoised = model.get_normalized_expression(return_mean=True)
388 adata.layers["totalVI_normalized"] = rna_norm
389 adata.obsm["protein_denoised"] = protein_denoised
391 # Clustering
392 sc.pp.neighbors(adata, use_rep="X_totalVI")
393 sc.tl.umap(adata)
394 sc.tl.leiden(adata)
396 return adata, model
398# Usage
399adata, model = analyze_citeseq(
400 adata_rna,
401 adata_protein,
402 batch_key="batch"
405# Visualize
406sc.pl.umap(adata, color=['leiden', 'batch'])
407```

Complete Pipeline

def analyze_citeseq(
    adata_rna,
    adata_protein,
    batch_key=None,
    n_top_genes=4000,
    n_latent=20
):
    """
    Complete CITE-seq analysis with totalVI.
    
    Parameters
    ----------
    adata_rna : AnnData
        RNA expression (raw counts)
    adata_protein : AnnData
        Protein expression (raw counts)
    batch_key : str, optional
        Batch column in obs
    n_top_genes : int
        Number of HVGs
    n_latent : int
        Latent dimensions
        
    Returns
    -------
    Tuple of (processed AnnData, trained model)
    """
    import scvi
    import scanpy as sc
    
    # Ensure same cells
    common_cells = adata_rna.obs_names.intersection(adata_protein.obs_names)
    adata = adata_rna[common_cells].copy()
    adata_protein = adata_protein[common_cells].copy()
    
    # Add protein to obsm
    adata.obsm["protein_expression"] = adata_protein.X.toarray() if hasattr(adata_protein.X, 'toarray') else adata_protein.X
    adata.uns["protein_names"] = list(adata_protein.var_names)
    
    # RNA QC
    # Handle both human (MT-) and mouse (mt-, Mt-) mitochondrial genes
    adata.var['mt'] = (
        adata.var_names.str.startswith('MT-') |
        adata.var_names.str.startswith('mt-') |
        adata.var_names.str.startswith('Mt-')
    )
    sc.pp.calculate_qc_metrics(adata, qc_vars=['mt'], inplace=True)
    adata = adata[adata.obs['pct_counts_mt'] < 20].copy()
    sc.pp.filter_genes(adata, min_cells=3)
    
    # Store counts
    adata.layers["counts"] = adata.X.copy()
    
    # HVG selection
    sc.pp.highly_variable_genes(
        adata,
        n_top_genes=n_top_genes,
        flavor="seurat_v3",
        batch_key=batch_key,
        layer="counts"
    )
    adata = adata[:, adata.var["highly_variable"]].copy()
    
    # Setup totalVI
    scvi.model.TOTALVI.setup_anndata(
        adata,
        layer="counts",
        protein_expression_obsm_key="protein_expression",
        batch_key=batch_key
    )
    
    # Train
    model = scvi.model.TOTALVI(adata, n_latent=n_latent)
    model.train(max_epochs=200, early_stopping=True)
    
    # Get representations
    adata.obsm["X_totalVI"] = model.get_latent_representation()
    rna_norm, protein_denoised = model.get_normalized_expression(return_mean=True)
    adata.layers["totalVI_normalized"] = rna_norm
    adata.obsm["protein_denoised"] = protein_denoised
    
    # Clustering
    sc.pp.neighbors(adata, use_rep="X_totalVI")
    sc.tl.umap(adata)
    sc.tl.leiden(adata)
    
    return adata, model

# Usage
adata, model = analyze_citeseq(
    adata_rna,
    adata_protein,
    batch_key="batch"
)

# Visualize
sc.pl.umap(adata, color=['leiden', 'batch'])
Raw lines 307 to 408
409## Troubleshooting
411| Issue | Cause | Solution |
412|-------|-------|----------|
413| Protein signal noisy | Background not removed | Use get_normalized_expression with denoising |
414| Batch effects persist | Need batch_key | Ensure batch_key is specified |
415| Memory error | Too many genes | Reduce n_top_genes |
416| Poor protein clustering | Few proteins | Normal - totalVI uses RNA for structure |

Troubleshooting

IssueCauseSolution
Protein signal noisyBackground not removedUse get_normalized_expression with denoising
Batch effects persistNeed batch_keyEnsure batch_key is specified
Memory errorToo many genesReduce n_top_genes
Poor protein clusteringFew proteinsNormal - totalVI uses RNA for structure
Raw lines 409 to 417
418## Key References
420- Gayoso et al. (2021) "Joint probabilistic modeling of single-cell multi-omic data with totalVI"

Key References

  • Gayoso et al. (2021) "Joint probabilistic modeling of single-cell multi-omic data with totalVI"
Raw lines 418 to 420