Skills › Knowledge Work Plugins › scvi-tools › multiome_multivi.md
multiome_multivi.md
384 lines · bio-research/skills/scvi-tools/references/multiome_multivi.md at 2574e29
1# Multiome Analysis with MultiVI
3This reference covers joint RNA and ATAC-seq analysis from multiome experiments using MultiVI.
Multiome Analysis with MultiVI
This reference covers joint RNA and ATAC-seq analysis from multiome experiments using MultiVI.
Raw lines 1 to 45## Overview
7MultiVI is a deep generative model for analyzing multiome data (simultaneous RNA-seq and ATAC-seq from the same cells). It:
8- Learns a joint latent representation across modalities
9- Handles missing modalities (RNA-only or ATAC-only cells)
10- Enables batch correction across experiments
11- Supports imputation of missing modalities
Overview
MultiVI is a deep generative model for analyzing multiome data (simultaneous RNA-seq and ATAC-seq from the same cells). It:
- Learns a joint latent representation across modalities
- Handles missing modalities (RNA-only or ATAC-only cells)
- Enables batch correction across experiments
- Supports imputation of missing modalities
13## Prerequisites
15```python
16import scvi
17import scanpy as sc
18import mudata as md
19import numpy as np
21print(f"scvi-tools version: {scvi.__version__}")
22```
Prerequisites
import scvi
import scanpy as sc
import mudata as md
import numpy as np
print(f"scvi-tools version: {scvi.__version__}")Raw lines 13 to 23Data Formats
Raw lines 24 to 2526### Option 1: MuData (Recommended)
28```python
29# Load multiome data as MuData
30mdata = md.read("multiome.h5mu")
32# Structure:
33# mdata.mod['rna'] - AnnData with RNA counts
34# mdata.mod['atac'] - AnnData with ATAC counts
36print(f"RNA: {mdata.mod['rna'].shape}")
37print(f"ATAC: {mdata.mod['atac'].shape}")
38```
Option 1: MuData (Recommended)
# Load multiome data as MuData
mdata = md.read("multiome.h5mu")
# Structure:
# mdata.mod['rna'] - AnnData with RNA counts
# mdata.mod['atac'] - AnnData with ATAC counts
print(f"RNA: {mdata.mod['rna'].shape}")
print(f"ATAC: {mdata.mod['atac'].shape}")Raw lines 26 to 3940### Option 2: Separate AnnData Objects
42```python
43# Load separately
44adata_rna = sc.read_h5ad("rna.h5ad")
45adata_atac = sc.read_h5ad("atac.h5ad")
47# Ensure same cells in same order
48common_cells = adata_rna.obs_names.intersection(adata_atac.obs_names)
49adata_rna = adata_rna[common_cells].copy()
50adata_atac = adata_atac[common_cells].copy()
51```
Option 2: Separate AnnData Objects
# Load separately
adata_rna = sc.read_h5ad("rna.h5ad")
adata_atac = sc.read_h5ad("atac.h5ad")
# Ensure same cells in same order
common_cells = adata_rna.obs_names.intersection(adata_atac.obs_names)
adata_rna = adata_rna[common_cells].copy()
adata_atac = adata_atac[common_cells].copy()Raw lines 40 to 5253## Step 1: Prepare RNA Data
55```python
56# RNA preprocessing (standard scvi-tools pipeline)
57adata_rna = mdata.mod['rna'].copy()
59# Filter
60sc.pp.filter_cells(adata_rna, min_genes=200)
61sc.pp.filter_genes(adata_rna, min_cells=3)
63# Store counts
64adata_rna.layers["counts"] = adata_rna.X.copy()
66# HVG selection
67sc.pp.highly_variable_genes(
68 adata_rna,
69 n_top_genes=4000,
70 flavor="seurat_v3",
71 layer="counts",
72 batch_key="batch" # If multiple batches
73)
75# Subset to HVGs
76adata_rna = adata_rna[:, adata_rna.var['highly_variable']].copy()
77```
Step 1: Prepare RNA Data
# RNA preprocessing (standard scvi-tools pipeline)
adata_rna = mdata.mod['rna'].copy()
# Filter
sc.pp.filter_cells(adata_rna, min_genes=200)
sc.pp.filter_genes(adata_rna, min_cells=3)
# Store counts
adata_rna.layers["counts"] = adata_rna.X.copy()
# HVG selection
sc.pp.highly_variable_genes(
adata_rna,
n_top_genes=4000,
flavor="seurat_v3",
layer="counts",
batch_key="batch" # If multiple batches
)
# Subset to HVGs
adata_rna = adata_rna[:, adata_rna.var['highly_variable']].copy()Raw lines 53 to 7879## Step 2: Prepare ATAC Data
81```python
82# ATAC preprocessing
83adata_atac = mdata.mod['atac'].copy()
85# Filter peaks
86sc.pp.filter_genes(adata_atac, min_cells=10)
88# Binarize accessibility
89adata_atac.X = (adata_atac.X > 0).astype(np.float32)
91# Select top accessible peaks (if too many)
92if adata_atac.n_vars > 50000:
93 peak_accessibility = np.array(adata_atac.X.sum(axis=0)).flatten()
94 top_peaks = np.argsort(peak_accessibility)[-50000:]
95 adata_atac = adata_atac[:, top_peaks].copy()
97# Store in layer
98adata_atac.layers["counts"] = adata_atac.X.copy()
99```
Step 2: Prepare ATAC Data
# ATAC preprocessing
adata_atac = mdata.mod['atac'].copy()
# Filter peaks
sc.pp.filter_genes(adata_atac, min_cells=10)
# Binarize accessibility
adata_atac.X = (adata_atac.X > 0).astype(np.float32)
# Select top accessible peaks (if too many)
if adata_atac.n_vars > 50000:
peak_accessibility = np.array(adata_atac.X.sum(axis=0)).flatten()
top_peaks = np.argsort(peak_accessibility)[-50000:]
adata_atac = adata_atac[:, top_peaks].copy()
# Store in layer
adata_atac.layers["counts"] = adata_atac.X.copy()Raw lines 79 to 100101## Step 3: Create Combined MuData
103```python
104# Ensure matching cells
105common_cells = adata_rna.obs_names.intersection(adata_atac.obs_names)
106adata_rna = adata_rna[common_cells].copy()
107adata_atac = adata_atac[common_cells].copy()
109# Create MuData
110mdata = md.MuData({
111 "rna": adata_rna,
112 "atac": adata_atac
113})
115print(f"Combined multiome: {mdata.n_obs} cells")
116print(f"RNA features: {mdata.mod['rna'].n_vars}")
117print(f"ATAC features: {mdata.mod['atac'].n_vars}")
118```
Step 3: Create Combined MuData
# Ensure matching cells
common_cells = adata_rna.obs_names.intersection(adata_atac.obs_names)
adata_rna = adata_rna[common_cells].copy()
adata_atac = adata_atac[common_cells].copy()
# Create MuData
mdata = md.MuData({
"rna": adata_rna,
"atac": adata_atac
})
print(f"Combined multiome: {mdata.n_obs} cells")
print(f"RNA features: {mdata.mod['rna'].n_vars}")
print(f"ATAC features: {mdata.mod['atac'].n_vars}")Raw lines 101 to 119120## Step 4: Setup MultiVI
122```python
123# Setup MuData for MultiVI
124scvi.model.MULTIVI.setup_mudata(
125 mdata,
126 rna_layer="counts",
127 atac_layer="counts",
128 batch_key="batch", # Optional
129 modalities={
130 "rna_layer": "rna",
131 "batch_key": "rna",
132 "atac_layer": "atac"
133 }
134)
135```
Step 4: Setup MultiVI
# Setup MuData for MultiVI
scvi.model.MULTIVI.setup_mudata(
mdata,
rna_layer="counts",
atac_layer="counts",
batch_key="batch", # Optional
modalities={
"rna_layer": "rna",
"batch_key": "rna",
"atac_layer": "atac"
}
)Raw lines 120 to 136137## Step 5: Train MultiVI
139```python
140# Create model
141model = scvi.model.MULTIVI(
142 mdata,
143 n_latent=20,
144 n_layers_encoder=2,
145 n_layers_decoder=2
146)
148# Train
149model.train(
150 max_epochs=300,
151 early_stopping=True,
152 early_stopping_patience=10,
153 batch_size=128
154)
156# Check training
157model.history['elbo_train'].plot()
158```
Step 5: Train MultiVI
# Create model
model = scvi.model.MULTIVI(
mdata,
n_latent=20,
n_layers_encoder=2,
n_layers_decoder=2
)
# Train
model.train(
max_epochs=300,
early_stopping=True,
early_stopping_patience=10,
batch_size=128
)
# Check training
model.history['elbo_train'].plot()Raw lines 137 to 159160## Step 6: Get Joint Representation
162```python
163# Latent representation
164latent = model.get_latent_representation()
166# Add to MuData
167mdata.obsm["X_MultiVI"] = latent
169# Clustering on joint space
170sc.pp.neighbors(mdata, use_rep="X_MultiVI")
171sc.tl.umap(mdata)
172sc.tl.leiden(mdata, resolution=1.0)
174# Visualize
175sc.pl.umap(mdata, color=['leiden', 'batch'], ncols=2)
176```
Step 6: Get Joint Representation
# Latent representation
latent = model.get_latent_representation()
# Add to MuData
mdata.obsm["X_MultiVI"] = latent
# Clustering on joint space
sc.pp.neighbors(mdata, use_rep="X_MultiVI")
sc.tl.umap(mdata)
sc.tl.leiden(mdata, resolution=1.0)
# Visualize
sc.pl.umap(mdata, color=['leiden', 'batch'], ncols=2)Raw lines 160 to 177Step 7: Modality-Specific Analysis
Raw lines 178 to 179180### Impute Missing Modality
182```python
183# Impute RNA expression for ATAC-only cells
184# (Useful when integrating with ATAC-only datasets)
185imputed_rna = model.get_normalized_expression(
186 modality="rna"
187)
189# Impute accessibility for RNA-only cells
190imputed_atac = model.get_accessibility_estimates()
191```
Impute Missing Modality
# Impute RNA expression for ATAC-only cells
# (Useful when integrating with ATAC-only datasets)
imputed_rna = model.get_normalized_expression(
modality="rna"
)
# Impute accessibility for RNA-only cells
imputed_atac = model.get_accessibility_estimates()Raw lines 180 to 192193### Differential Analysis
195```python
196# Differential expression (RNA)
197de_results = model.differential_expression(
198 groupby="leiden",
199 group1="0",
200 group2="1"
201)
203# Differential accessibility (ATAC)
204da_results = model.differential_accessibility(
205 groupby="leiden",
206 group1="0",
207 group2="1"
208)
209```
Differential Analysis
# Differential expression (RNA)
de_results = model.differential_expression(
groupby="leiden",
group1="0",
group2="1"
)
# Differential accessibility (ATAC)
da_results = model.differential_accessibility(
groupby="leiden",
group1="0",
group2="1"
)Raw lines 193 to 210211## Handling Partial Data
213MultiVI can integrate datasets with only one modality:
215```python
216# Dataset 1: Full multiome
217# Dataset 2: RNA only
218# Dataset 3: ATAC only
220# Mark missing modalities
221mdata.obs['modality'] = 'paired' # For cells with both
222# For RNA-only cells, ATAC data should be missing/NaN
223# For ATAC-only cells, RNA data should be missing/NaN
225# MultiVI handles this automatically during training
226```
Handling Partial Data
MultiVI can integrate datasets with only one modality:
# Dataset 1: Full multiome
# Dataset 2: RNA only
# Dataset 3: ATAC only
# Mark missing modalities
mdata.obs['modality'] = 'paired' # For cells with both
# For RNA-only cells, ATAC data should be missing/NaN
# For ATAC-only cells, RNA data should be missing/NaN
# MultiVI handles this automatically during trainingRaw lines 211 to 227228## Complete Pipeline
230```python
231def analyze_multiome(
232 adata_rna,
233 adata_atac,
234 batch_key=None,
235 n_top_genes=4000,
236 n_top_peaks=50000,
237 n_latent=20,
238 max_epochs=300
239):
240 """
241 Complete multiome analysis with MultiVI.
243 Parameters
244 ----------
245 adata_rna : AnnData
246 RNA count data
247 adata_atac : AnnData
248 ATAC peak data
249 batch_key : str, optional
250 Batch column name
251 n_top_genes : int
252 Number of HVGs for RNA
253 n_top_peaks : int
254 Number of top peaks for ATAC
255 n_latent : int
256 Latent dimensions
257 max_epochs : int
258 Maximum training epochs
260 Returns
261 -------
262 MuData with joint representation
263 """
264 import scvi
265 import scanpy as sc
266 import mudata as md
267 import numpy as np
269 # Get common cells
270 common_cells = adata_rna.obs_names.intersection(adata_atac.obs_names)
271 adata_rna = adata_rna[common_cells].copy()
272 adata_atac = adata_atac[common_cells].copy()
274 # RNA preprocessing
275 sc.pp.filter_genes(adata_rna, min_cells=3)
276 adata_rna.layers["counts"] = adata_rna.X.copy()
278 if batch_key:
279 sc.pp.highly_variable_genes(
280 adata_rna, n_top_genes=n_top_genes,
281 flavor="seurat_v3", layer="counts", batch_key=batch_key
282 )
283 else:
284 sc.pp.normalize_total(adata_rna, target_sum=1e4)
285 sc.pp.log1p(adata_rna)
286 sc.pp.highly_variable_genes(adata_rna, n_top_genes=n_top_genes)
287 adata_rna.X = adata_rna.layers["counts"].copy()
289 adata_rna = adata_rna[:, adata_rna.var['highly_variable']].copy()
291 # ATAC preprocessing
292 sc.pp.filter_genes(adata_atac, min_cells=10)
293 adata_atac.X = (adata_atac.X > 0).astype(np.float32)
295 if adata_atac.n_vars > n_top_peaks:
296 peak_acc = np.array(adata_atac.X.sum(axis=0)).flatten()
297 top_idx = np.argsort(peak_acc)[-n_top_peaks:]
298 adata_atac = adata_atac[:, top_idx].copy()
300 adata_atac.layers["counts"] = adata_atac.X.copy()
302 # Create MuData
303 mdata = md.MuData({"rna": adata_rna, "atac": adata_atac})
305 # Setup and train
306 scvi.model.MULTIVI.setup_mudata(
307 mdata,
308 rna_layer="counts",
309 atac_layer="counts",
310 batch_key=batch_key,
311 modalities={"rna_layer": "rna", "batch_key": "rna", "atac_layer": "atac"}
312 )
314 model = scvi.model.MULTIVI(mdata, n_latent=n_latent)
315 model.train(max_epochs=max_epochs, early_stopping=True)
317 # Add representation
318 mdata.obsm["X_MultiVI"] = model.get_latent_representation()
320 # Cluster
321 sc.pp.neighbors(mdata, use_rep="X_MultiVI")
322 sc.tl.umap(mdata)
323 sc.tl.leiden(mdata)
325 return mdata, model
328# Usage
329mdata, model = analyze_multiome(
330 adata_rna,
331 adata_atac,
332 batch_key="sample"
333)
335sc.pl.umap(mdata, color=['leiden', 'sample'])
336```
Complete Pipeline
def analyze_multiome(
adata_rna,
adata_atac,
batch_key=None,
n_top_genes=4000,
n_top_peaks=50000,
n_latent=20,
max_epochs=300
):
"""
Complete multiome analysis with MultiVI.
Parameters
----------
adata_rna : AnnData
RNA count data
adata_atac : AnnData
ATAC peak data
batch_key : str, optional
Batch column name
n_top_genes : int
Number of HVGs for RNA
n_top_peaks : int
Number of top peaks for ATAC
n_latent : int
Latent dimensions
max_epochs : int
Maximum training epochs
Returns
-------
MuData with joint representation
"""
import scvi
import scanpy as sc
import mudata as md
import numpy as np
# Get common cells
common_cells = adata_rna.obs_names.intersection(adata_atac.obs_names)
adata_rna = adata_rna[common_cells].copy()
adata_atac = adata_atac[common_cells].copy()
# RNA preprocessing
sc.pp.filter_genes(adata_rna, min_cells=3)
adata_rna.layers["counts"] = adata_rna.X.copy()
if batch_key:
sc.pp.highly_variable_genes(
adata_rna, n_top_genes=n_top_genes,
flavor="seurat_v3", layer="counts", batch_key=batch_key
)
else:
sc.pp.normalize_total(adata_rna, target_sum=1e4)
sc.pp.log1p(adata_rna)
sc.pp.highly_variable_genes(adata_rna, n_top_genes=n_top_genes)
adata_rna.X = adata_rna.layers["counts"].copy()
adata_rna = adata_rna[:, adata_rna.var['highly_variable']].copy()
# ATAC preprocessing
sc.pp.filter_genes(adata_atac, min_cells=10)
adata_atac.X = (adata_atac.X > 0).astype(np.float32)
if adata_atac.n_vars > n_top_peaks:
peak_acc = np.array(adata_atac.X.sum(axis=0)).flatten()
top_idx = np.argsort(peak_acc)[-n_top_peaks:]
adata_atac = adata_atac[:, top_idx].copy()
adata_atac.layers["counts"] = adata_atac.X.copy()
# Create MuData
mdata = md.MuData({"rna": adata_rna, "atac": adata_atac})
# Setup and train
scvi.model.MULTIVI.setup_mudata(
mdata,
rna_layer="counts",
atac_layer="counts",
batch_key=batch_key,
modalities={"rna_layer": "rna", "batch_key": "rna", "atac_layer": "atac"}
)
model = scvi.model.MULTIVI(mdata, n_latent=n_latent)
model.train(max_epochs=max_epochs, early_stopping=True)
# Add representation
mdata.obsm["X_MultiVI"] = model.get_latent_representation()
# Cluster
sc.pp.neighbors(mdata, use_rep="X_MultiVI")
sc.tl.umap(mdata)
sc.tl.leiden(mdata)
return mdata, model
# Usage
mdata, model = analyze_multiome(
adata_rna,
adata_atac,
batch_key="sample"
)
sc.pl.umap(mdata, color=['leiden', 'sample'])Raw lines 228 to 337338## Peak-to-Gene Linking
340```python
341# Link ATAC peaks to genes based on correlation in latent space
342# This identifies regulatory relationships
344def link_peaks_to_genes(model, mdata, distance_threshold=100000):
345 """
346 Link peaks to nearby genes based on correlation.
348 Parameters
349 ----------
350 model : MULTIVI
351 Trained model
352 mdata : MuData
353 Multiome data
354 distance_threshold : int
355 Maximum distance (bp) to link peak to gene
357 Returns
358 -------
359 DataFrame of peak-gene links
360 """
361 # Get imputed values
362 rna_imputed = model.get_normalized_expression()
363 atac_imputed = model.get_accessibility_estimates()
365 # Correlate peak accessibility with gene expression
366 # for peaks near gene promoters
367 # ... (requires genomic coordinates)
369 return peak_gene_links
370```
Peak-to-Gene Linking
# Link ATAC peaks to genes based on correlation in latent space
# This identifies regulatory relationships
def link_peaks_to_genes(model, mdata, distance_threshold=100000):
"""
Link peaks to nearby genes based on correlation.
Parameters
----------
model : MULTIVI
Trained model
mdata : MuData
Multiome data
distance_threshold : int
Maximum distance (bp) to link peak to gene
Returns
-------
DataFrame of peak-gene links
"""
# Get imputed values
rna_imputed = model.get_normalized_expression()
atac_imputed = model.get_accessibility_estimates()
# Correlate peak accessibility with gene expression
# for peaks near gene promoters
# ... (requires genomic coordinates)
return peak_gene_linksRaw lines 338 to 371372## Troubleshooting
374| Issue | Cause | Solution |
375|-------|-------|----------|
376| Different cell counts | Cells missing in one modality | Use common cells only |
377| Training instability | Imbalanced modalities | Normalize feature counts |
378| Poor clustering | Too few features | Increase n_top_genes/peaks |
379| Memory error | Large ATAC matrix | Reduce peak count, use sparse |
380| Batch dominates | Strong technical effects | Ensure batch_key is set |
Troubleshooting
| Issue | Cause | Solution |
|---|---|---|
| Different cell counts | Cells missing in one modality | Use common cells only |
| Training instability | Imbalanced modalities | Normalize feature counts |
| Poor clustering | Too few features | Increase n_top_genes/peaks |
| Memory error | Large ATAC matrix | Reduce peak count, use sparse |
| Batch dominates | Strong technical effects | Ensure batch_key is set |
382## Key References
384- Ashuach et al. (2023) "MultiVI: deep generative model for the integration of multimodal data"
Key References
- Ashuach et al. (2023) "MultiVI: deep generative model for the integration of multimodal data"