pertpy.tools.HDBSCANSpace#

class HDBSCANSpace[source]#

Cluster the given data using HDBSCAN.

Methods table#

add(adata, *, perturbations[, ...])

Add perturbations linearly.

compute(adata, *[, layer_key, ...])

Computes a clustering using hierarchical density-based spatial clustering of applications (HDBSCAN).

compute_control_diff(adata, *[, target_col, ...])

Subtract mean of the control from the perturbation.

dose_response(adata, *[, target_col, ...])

Quantify the effect size of each perturbation as a function of dose.

evaluate_clustering(adata, true_label_col, ...)

Evaluation of previously computed clustering against ground truth labels.

evaluate_combinations(adata, *[, ...])

Score how well an additive model predicts combination perturbations.

fit_dose_response(adata[, response, ...])

Fit a four-parameter Hill curve for each perturbation.

label_transfer(adata, *[, target_column, ...])

Impute missing values in the specified column using KNN imputation in the space defined by use_rep.

nearest_perturbations(adata, perturbation, *)

Rank perturbations by their proximity to a query perturbation in a perturbation space.

plot_dose_response(adata[, response, ...])

Plot the measured responses and fitted Hill curves of each perturbation.

plot_similarity(adata, *[, target_col, ...])

Plot a clustered heatmap of pairwise distances between perturbations.

subtract(adata, *, perturbations[, ...])

Subtract perturbations linearly.

Methods#

add(adata, *, perturbations, reference_key='control', ensure_consistency=True, target_col='perturbation')#

Add perturbations linearly. Assumes input of size n_perts x dimensionality.

Parameters:
adata AnnData

Anndata object of size n_perts x dim.

perturbations Iterable[str]

Perturbations to add.

reference_key str, default: 'control'

perturbation source from which the perturbation summation starts.

ensure_consistency bool, default: True

If True, differentiate against control via compute_control_diff before combining so that “perturbation - perturbation == control” holds in the resulting space. Set False only if the input has already been differenced.

target_col str, default: 'perturbation'

.obs column name that stores the label of the perturbation applied to each cell.

Return type:

tuple[AnnData, AnnData] | AnnData

Returns:

Anndata object of size (n_perts+1) x dim, where the last row is the addition of the specified perturbations. If ensure_consistency is True, returns a tuple of (new_perturbation, adata) where adata is the AnnData object provided as input but updated using compute_control_diff.

Examples

Example usage with PseudobulkSpace:

>>> import pertpy as pt
>>> mdata = pt.ds.papalexi_2021()
>>> ps = pt.tl.PseudobulkSpace()
>>> ps_adata = ps.compute(mdata["rna"], target_col="gene_target", groups_col="gene_target")
>>> new_perturbation = ps.add(ps_adata, perturbations=["ATF2", "CD86"], reference_key="NT")
compute(adata, *, layer_key=None, embedding_key=None, cluster_key='hdbscan', copy=True, return_object=False, **kwargs)[source]#

Computes a clustering using hierarchical density-based spatial clustering of applications (HDBSCAN).

HDBSCAN extends DBSCAN by converting it into a hierarchical clustering algorithm, removing the need to pick a single density threshold (eps) and handling clusters of varying density.

Parameters:
adata AnnData

Anndata object of size cells x genes

layer_key str | None, default: None

If specified and exists in the adata, the clustering is done by using it. Otherwise, clustering is done with .X.

embedding_key str | None, default: None

if specified and exists in the adata, the clustering is done with that embedding. Otherwise, clustering is done with .X.

cluster_key str, default: 'hdbscan'

name of the .obs column to store the cluster labels.

copy bool, default: True

if True returns a new Anndata of same size with the new column; otherwise it updates the initial adata

return_object bool, default: False

if True returns the clustering object

**kwargs

Are passed to sklearn’s HDBSCAN.

Return type:

tuple[AnnData, object] | AnnData

Returns:

If return_object is True, the adata and the clustering object is returned. Otherwise, only the adata is returned. The adata is updated with a new .obs column as specified in cluster_key, that stores the cluster labels.

Examples

>>> import pertpy as pt
>>> mdata = pt.ds.papalexi_2021()
>>> hdbscan = pt.tl.HDBSCANSpace()
>>> hdbscan_adata = hdbscan.compute(mdata["rna"])
compute_control_diff(adata, *, target_col='perturbation', group_col=None, reference_key='control', layer_key=None, new_layer_key='control_diff', embedding_key=None, new_embedding_key='control_diff', all_data=False, copy=True)#

Subtract mean of the control from the perturbation.

Parameters:
adata AnnData

Anndata object of size cells x genes.

target_col str, default: 'perturbation'

.obs column name that stores the label of the perturbation applied to each cell.

group_col str | None, default: None

.obs column name that stores the label of the group of each cell. If None, ignore groups.

reference_key str, default: 'control'

The key of the control values.

layer_key str | None, default: None

Key of the AnnData layer to use for computation.

new_layer_key str, default: 'control_diff'

the results are stored in the given layer.

embedding_key str | None, default: None

obsm key of the AnnData embedding to use for computation.

new_embedding_key str, default: 'control_diff'

Results are stored in a new embedding in obsm with this key.

all_data bool, default: False

if True, do the computation in all data representations (X, all layers and all embeddings)

copy bool, default: True

If True returns a new AnnData; otherwise updates the input AnnData in place.

Return type:

AnnData

Returns:

Updated AnnData object.

Examples

Example usage with PseudobulkSpace:

>>> import pertpy as pt
>>> mdata = pt.ds.papalexi_2021()
>>> ps = pt.tl.PseudobulkSpace()
>>> diff_adata = ps.compute_control_diff(mdata["rna"], target_col="gene_target", reference_key="NT")
dose_response(adata, *, target_col='perturbation', dose_col='dose', reference_key='control', metric='edistance', layer_key=None, embedding_key=None, **kwargs)#

Quantify the effect size of each perturbation as a function of dose.

For every (perturbation, dose) group the statistical distance to reference_key is computed in the chosen representation using Distance. Operates on cell-level data, since distances are defined between groups of cells.

Parameters:
adata AnnData

Cell-level AnnData.

target_col str, default: 'perturbation'

.obs column with the perturbation label.

dose_col str, default: 'dose'

.obs column with the (numeric) dose.

reference_key str, default: 'control'

Control perturbation all doses are compared against.

metric Literal['edistance', 'euclidean', 'root_mean_squared_error', 'mse', 'mean_absolute_error', 'pearson_distance', 'spearman_distance', 'kendalltau_distance', 'cosine_distance', 'r2_distance', 'mean_pairwise', 'mmd', 'wasserstein', 'sym_kldiv', 't_test', 'ks_test', 'nb_ll', 'classifier_proba', 'classifier_cp', 'mean_var_distribution', 'mahalanobis'], default: 'edistance'

Distance metric passed to Distance.

layer_key str | None, default: None

Layer to compute distances and mean expression from.

embedding_key str | None, default: None

.obsm embedding to compute distances from.

kwargs

Passed to onesided_distances().

Return type:

AnnData

Returns:

AnnData with one observation per non-reference (perturbation, dose) group, holding the mean expression in X and the distance in .obs.

Examples

>>> import pertpy as pt
>>> adata = pt.ds.srivatsan_2020_sciplex2()
>>> ps = pt.tl.PseudobulkSpace()
>>> dose_adata = ps.dose_response(adata, dose_col="dose_value", embedding_key="X_pca")
evaluate_clustering(adata, true_label_col, cluster_col, metrics=None, *, layer_key=None, embedding_key=None, **kwargs)#

Evaluation of previously computed clustering against ground truth labels.

Parameters:
adata AnnData

AnnData object that contains the clustered data and the cluster labels.

true_label_col str

ground truth labels.

cluster_col str

cluster computed labels.

metrics Iterable[str] | None, default: None

Metrics to compute. If None it defaults to ["nmi", "ari", "asw"] — the canonical trio for clustering benchmarks (mutual information, agreement, silhouette).

layer_key str | None, default: None

Layer to resolve cell coordinates from when computing ASW.

embedding_key str | None, default: None

Embedding to resolve cell coordinates from when computing ASW.

**kwargs

Additional arguments to pass to the metrics. For nmi, average_method can be passed. For asw, metric, distances, sample_size, and random_state can be passed.

Examples

Example usage with KMeansSpace:

>>> import pertpy as pt
>>> mdata = pt.ds.papalexi_2021()
>>> kmeans = pt.tl.KMeansSpace()
>>> kmeans_adata = kmeans.compute(mdata["rna"], n_clusters=26)
>>> results = kmeans.evaluate_clustering(
...     kmeans_adata, true_label_col="gene_target", cluster_col="k-means", metrics=["nmi"]
... )
evaluate_combinations(adata, *, combinations=None, target_col='perturbation', reference_key='control', metric='pearson', sep='+')#

Score how well an additive model predicts combination perturbations.

For every combination "A+B" whose components A and B are each present as single perturbations, the additive prediction effect(A) + effect(B) is compared against the measured combination effect, where effects are taken relative to reference_key. A small distance indicates additive, non-interacting perturbations, whereas a large deviation flags a genetic or pharmacological interaction.

Parameters:
adata AnnData

Perturbation-level AnnData indexed by perturbation (one observation per perturbation).

combinations Iterable[str] | None, default: None

Combination names to evaluate. If None, every obs_name containing sep whose components are all present as singles is used.

target_col str, default: 'perturbation'

.obs column identifying each perturbation.

reference_key str, default: 'control'

Control perturbation subtracted to obtain effects. If absent, values are used as-is.

metric Literal['pearson', 'cosine', 'euclidean'], default: 'pearson'

Distance between predicted and measured combination effects.

sep str, default: '+'

Separator between components in combination names.

Return type:

DataFrame

Returns:

DataFrame indexed by combination with distance, predicted_magnitude and measured_magnitude columns, sorted by ascending distance.

Examples

>>> import pertpy as pt
>>> adata = pt.ds.norman_2019()
>>> ps = pt.tl.PseudobulkSpace()
>>> ps_adata = ps.compute(adata, target_col="perturbation_name")
>>> scores = ps.evaluate_combinations(ps_adata, target_col="perturbation_name", reference_key="control")
fit_dose_response(adata, response='distance', *, target_col='perturbation', dose_col='dose', key_added=None, n_jobs=None)#

Fit a four-parameter Hill curve for each perturbation.

Perturbations whose curve cannot be fit get a warning and NaN parameters.

Parameters:
adata AnnData

AnnData with one observation per dose or replicate, such as the output of dose_response().

response str, default: 'distance'

.obs column or gene in var_names to fit.

target_col str, default: 'perturbation'

.obs column identifying the perturbation.

dose_col str, default: 'dose'

.obs column containing non-negative numeric doses.

key_added str | None, default: None

Prefix of the .obs columns the results are written to. Defaults to response.

n_jobs int | None, default: None

Number of jobs to distribute the perturbations over, passed to joblib.Parallel. None means one job, -1 all available cores.

Return type:

None

Returns:

Adds the fitted response and the per-perturbation e0, emax, slope, ec50, ec50_se, r_squared and ec50_in_range to .obs, prefixed with key_added.

Examples

>>> import pertpy as pt
>>> adata = pt.ds.srivatsan_2020_sciplex2()
>>> ps = pt.tl.PseudobulkSpace()
>>> dose_adata = ps.dose_response(adata, dose_col="dose_value", embedding_key="X_pca")
>>> ps.fit_dose_response(dose_adata, dose_col="dose_value")
>>> ps.fit_dose_response(dose_adata, "CDKN1A", dose_col="dose_value")
label_transfer(adata, *, target_column='perturbation', column_uncertainty_score_key='perturbation_transfer_uncertainty', target_val='unknown', neighbors_key='neighbors')#

Impute missing values in the specified column using KNN imputation in the space defined by use_rep.

Uncertainty is calculated as the entropy of the label distribution in the neighborhood of the target cell. In other words, a cell where all neighbors have the same set of labels will have an uncertainty of 0, whereas a cell where all neighbors have many different labels will have high uncertainty.

Parameters:
adata AnnData

The AnnData object containing single-cell data.

target_column str, default: 'perturbation'

The column name in adata.obs to perform imputation on.

column_uncertainty_score_key str, default: 'perturbation_transfer_uncertainty'

The column name in adata.obs to store the uncertainty score of the label transfer.

target_val str, default: 'unknown'

The target value to impute.

neighbors_key str, default: 'neighbors'

The key in adata.uns where the neighbors are stored.

Return type:

None

Examples

>>> import pertpy as pt
>>> import scanpy as sc
>>> import numpy as np
>>> adata = sc.datasets.pbmc68k_reduced()
>>> # randomly dropout 10% of the data annotations
>>> adata.obs["perturbation"] = adata.obs["louvain"].astype(str).copy()
>>> random_cells = np.random.choice(adata.obs.index, int(adata.obs.shape[0] * 0.1), replace=False)
>>> adata.obs.loc[random_cells, "perturbation"] = "unknown"
>>> sc.pp.neighbors(adata)
>>> sc.tl.umap(adata)
>>> ps = pt.tl.PseudobulkSpace()
>>> ps.label_transfer(adata)
nearest_perturbations(adata, perturbation, *, target_col='perturbation', n_neighbors=10, layer_key=None, embedding_key=None, metric='euclidean')#

Rank perturbations by their proximity to a query perturbation in a perturbation space.

Operates on a perturbation-level AnnData (one observation per perturbation), i.e. the output of any compute. Useful for discovering perturbations with a similar mechanism of action. If adata.obsp["distances"] is present (as produced by DistanceSpace) and no representation is requested explicitly, those precomputed distances are used directly.

Parameters:
adata AnnData

Perturbation-level AnnData indexed by perturbation.

perturbation str

The query perturbation to find neighbors for.

target_col str, default: 'perturbation'

.obs column identifying each perturbation.

n_neighbors int, default: 10

Number of nearest perturbations to return.

layer_key str | None, default: None

Layer to compute distances from.

embedding_key str | None, default: None

.obsm embedding to compute distances from.

metric str, default: 'euclidean'

Distance metric passed to sklearn.metrics.pairwise_distances().

Return type:

DataFrame

Returns:

DataFrame indexed by perturbation with a distance column, sorted ascending and excluding the query.

Examples

>>> import pertpy as pt
>>> adata = pt.ds.norman_2019()
>>> ps = pt.tl.PseudobulkSpace()
>>> ps_adata = ps.compute(adata, target_col="perturbation_name")
>>> neighbors = ps.nearest_perturbations(ps_adata, "CBL+CNN1", target_col="perturbation_name")
plot_dose_response(adata, response='distance', *, target_col='perturbation', dose_col='dose', key_added=None, perturbations=None, ncols=4, return_fig=False)#

Plot the measured responses and fitted Hill curves of each perturbation.

Parameters:
adata AnnData

AnnData with one observation per dose or replicate, such as the output of dose_response().

response str, default: 'distance'

.obs column or gene in var_names to plot.

target_col str, default: 'perturbation'

.obs column identifying the perturbation.

dose_col str, default: 'dose'

.obs column containing the doses.

key_added str | None, default: None

Prefix passed to fit_dose_response(). Defaults to response.

perturbations Sequence[str] | None, default: None

Perturbations to plot. Defaults to all.

ncols int, default: 4

Number of panels per row.

return_fig bool, default: False

if True, returns figure of the plot, that can be used for saving.

Return type:

Figure | None

Returns:

If return_fig is True, returns the figure, otherwise None.

Examples

>>> import pertpy as pt
>>> adata = pt.ds.srivatsan_2020_sciplex2()
>>> ps = pt.tl.PseudobulkSpace()
>>> dose_adata = ps.dose_response(adata, dose_col="dose_value", embedding_key="X_pca")
>>> ps.fit_dose_response(dose_adata, dose_col="dose_value")
>>> ps.plot_dose_response(dose_adata, dose_col="dose_value")
Preview:
../../_images/dose_response.png
plot_similarity(adata, *, target_col='perturbation', layer_key=None, embedding_key=None, metric='euclidean', cmap='viridis', **kwargs)#

Plot a clustered heatmap of pairwise distances between perturbations.

Uses adata.obsp["distances"] when present (e.g. from DistanceSpace) and otherwise computes pairwise distances in the chosen representation.

Parameters:
adata AnnData

Perturbation-level AnnData indexed by perturbation.

target_col str, default: 'perturbation'

.obs column identifying each perturbation.

layer_key str | None, default: None

Layer to compute distances from.

embedding_key str | None, default: None

.obsm embedding to compute distances from.

metric str, default: 'euclidean'

Distance metric passed to sklearn.metrics.pairwise_distances().

cmap str, default: 'viridis'

Matplotlib colormap.

kwargs

Passed to seaborn.clustermap().

Returns:

The grid returned by seaborn.clustermap().

Examples

>>> import pertpy as pt
>>> adata = pt.ds.norman_2019()
>>> ds = pt.tl.DistanceSpace()
>>> ds_adata = ds.compute(adata, target_col="perturbation_name", metric="edistance")
>>> ds.plot_similarity(ds_adata, target_col="perturbation_name")
subtract(adata, *, perturbations, reference_key='control', ensure_consistency=True, target_col='perturbation')#

Subtract perturbations linearly. Assumes input of size n_perts x dimensionality.

Parameters:
adata AnnData

Anndata object of size n_perts x dim.

perturbations Iterable[str]

Perturbations to subtract.

reference_key str, default: 'control'

Perturbation source from which the perturbation subtraction starts.

ensure_consistency bool, default: True

If True, differentiate against control via compute_control_diff before combining so that “perturbation - perturbation == control” holds in the resulting space. Set False only if the input has already been differenced.

target_col str, default: 'perturbation'

.obs column name that stores the label of the perturbation applied to each cell.

Return type:

tuple[AnnData, AnnData] | AnnData

Returns:

Anndata object of size (n_perts+1) x dim, where the last row is the subtraction of the specified perturbations. If ensure_consistency is True, returns a tuple of (new_perturbation, adata) where adata is the AnnData object provided as input but updated using compute_control_diff.

Examples

Example usage with PseudobulkSpace:

>>> import pertpy as pt
>>> mdata = pt.ds.papalexi_2021()
>>> ps = pt.tl.PseudobulkSpace()
>>> ps_adata = ps.compute(mdata["rna"], target_col="gene_target", groups_col="gene_target")
>>> new_perturbation = ps.subtract(ps_adata, reference_key="ATF2", perturbations=["BRD4", "CUL3"])