GPU Training with Sinkhorn#

How to speed up GGML training by using the Sinkhorn OT solver on a GPU.

[1]:
import ggml_ot
import pandas as pd

By default, GGML uses the exact Earth Mover’s Distance (EMD). EMD calls a network-simplex LP solver per distribution pair which is only supported on CPU. The Sinkhorn solver instead expresses OT as a sequence of batched matrix operations that vectorize across all pairs in a triplet batch, which maps cleanly onto a GPU.

Setup dataset with normalized supports#

Sinkhorn uses entropic regularization which is controlled by the entropic_reg parameter and is sensitive to the scale of the data. To better compare and tune entropic_reg across datasets, we load the shortcut with normalize=True.

We reuse the Breast cancer dataset from Kumar et al., 2023 (714,331 cells from 126 donors) from the fast approximation tutorial.

[2]:
# Wrapper around load_cellxgene(...) + from_anndata(...) for this example dataset.
# See the dataset-loading tutorial for using your own AnnData object.
dataset = ggml_ot.data.breastcancer(n_top_genes=2000, normalize=True, n_cells=100)
[ggml_ot] Neither `path=` nor `ggml_ot.settings.datadir` is set; caching dataset under /home/kuehn/ot_metric_learning/gaussian-ground-metric-learning/code/ggml-ot_privat/docs/source/tutorials/data. Set `ggml_ot.settings.datadir = ...` or pass `path=` to override.
[3]:
scores = {}

Sinkhorn on GPU#

Passing entropic_reg > 0 to train() dispatches the inner OT solve to Sinkhorn. With settings.device="cuda" the GGML optimization, including the inner OT, is performed on the GPU as batched vectorized operations which leads to a significant speed-up.

[4]:
ggml_ot.settings.device = "cuda"

_, scores["sinkhorn_gpu"] = dataset.train_test(entropic_reg=5.0, n_splits=1)
../_images/tutorials_4_gpu_sinkhorn_7_0.png
metric knn ari epoch_time(s)
mean±SD mean SD mean SD mean SD
0 0.84 0.00 0.76 0.00 0.09 0.00

Sinkhorn on CPU#

Even on CPU (settings.device="cpu") the computation time benefits from batched tensor operations and better utilization of large number of CPU threads (settings.n_threads) compared to the slow training using EMD2 (entropic_reg=0).

[5]:
ggml_ot.settings.device = "cpu"
ggml_ot.settings.n_threads = 64

_, scores["sinkhorn_cpu"] = dataset.train_test(entropic_reg=5.0, n_splits=1)
../_images/tutorials_4_gpu_sinkhorn_9_0.png
metric knn ari epoch_time(s)
mean±SD mean SD mean SD mean SD
0 0.84 0.00 0.70 0.00 1.09 0.00

Exact EMD2 on GPU and CPU#

For contrast, we run the exact EMD2 solver (entropic_reg=0.0, the default) on both devices. EMD2 is computed by the network-simplex LP solver shipped with POT, which is not batched and runs on the CPU. Setting settings.device="cuda" therefore only moves tensors to the GPU for the triplet loss, the OT solves themselves stay on the CPU and train() emits a corresponding notification.

[6]:
ggml_ot.settings.device = "cuda"

_, scores["emd2_gpu"] = dataset.train_test(entropic_reg=0.0, n_splits=1)
[ggml_ot] Exact EMD2 solver is CPU-bound, to leverage CUDA use Sinkhorn solver (pass parameter entropic_reg > 0).
../_images/tutorials_4_gpu_sinkhorn_11_1.png
metric knn ari epoch_time(s)
mean±SD mean SD mean SD mean SD
0 0.84 0.00 0.65 0.00 144.35 0.00
[7]:
ggml_ot.settings.device = "cpu"

# entropic_reg=0.0 (emd2) is default
_, scores["emd2_cpu"] = dataset.train_test(n_splits=1)
../_images/tutorials_4_gpu_sinkhorn_12_0.png
metric knn ari epoch_time(s)
mean±SD mean SD mean SD mean SD
0 0.84 0.00 0.75 0.00 127.38 0.00

As expected, the non-batched EMD2 cannot leverage GPU speed-up and massive parallelization.

Performance comparison#

train_test() reports the mean epoch time alongside the usual classification and clustering metrics. We sort by epoch time and show the four setups from fastest to slowest.

[8]:
scores_df = (
    pd.concat([scores[setup] for setup in scores], ignore_index=True)
    .set_index(pd.Index(scores.keys(), name="setup"))
    .sort_values(("epoch_time(s)", "mean"))
)

ggml_ot.pl.table(
    scores_df,
    style_performance=True,
    title="Sinkhorn vs. EMD2 on GPU and CPU",
)
Sinkhorn vs. EMD2 on GPU and CPU
metric knn ari epoch_time(s)
mean±SD mean SD mean SD mean SD
setup            
sinkhorn_gpu 0.84 0.00 0.76 0.00 0.09 0.00
sinkhorn_cpu 0.84 0.00 0.70 0.00 1.09 0.00
emd2_cpu 0.84 0.00 0.75 0.00 127.38 0.00
emd2_gpu 0.84 0.00 0.65 0.00 144.35 0.00

Takeaways:

  • To leverage GPU speed-up the Sinkhorn solver must be used with entropic_reg > 0

    • normalize() before tuning entropic_reg makes regularization strengths transferable across datasets

  • EMD2 solver is not batched and computations are CPU-bound regardless of settings.device

See the fast approximations tutorial for other computational speed-ups for both, EMD2 and Sinkhorn.

Note

To improve training time also consider other speed-up strategies like PCA embeddings, cell-type grouping, and cell subsampling, see Tutorial on Computational Speed-ups.