GPU Training with Sinkhorn#
How to speed up GGML training by using the Sinkhorn OT solver on a GPU.
[1]:
import ggml_ot
import pandas as pd
By default, GGML uses the exact Earth Mover’s Distance (EMD). EMD calls a network-simplex LP solver per distribution pair which is only supported on CPU. The Sinkhorn solver instead expresses OT as a sequence of batched matrix operations that vectorize across all pairs in a triplet batch, which maps cleanly onto a GPU.
Setup dataset with normalized supports#
Sinkhorn uses entropic regularization which is controlled by the entropic_reg parameter and is sensitive to the scale of the data. To better compare and tune entropic_reg across datasets, we load the shortcut with normalize=True.
We reuse the Breast cancer dataset from Kumar et al., 2023 (714,331 cells from 126 donors) from the fast approximation tutorial.
[2]:
# Wrapper around load_cellxgene(...) + from_anndata(...) for this example dataset.
# See the dataset-loading tutorial for using your own AnnData object.
dataset = ggml_ot.data.breastcancer(n_top_genes=2000, normalize=True, n_cells=100)
[ggml_ot] Neither `path=` nor `ggml_ot.settings.datadir` is set; caching dataset under /home/kuehn/ot_metric_learning/gaussian-ground-metric-learning/code/ggml-ot_privat/docs/source/tutorials/data. Set `ggml_ot.settings.datadir = ...` or pass `path=` to override.
[3]:
scores = {}
Sinkhorn on GPU#
Passing entropic_reg > 0 to train() dispatches the inner OT solve to Sinkhorn. With settings.device="cuda" the GGML optimization, including the inner OT, is performed on the GPU as batched vectorized operations which leads to a significant speed-up.
[4]:
ggml_ot.settings.device = "cuda"
_, scores["sinkhorn_gpu"] = dataset.train_test(entropic_reg=5.0, n_splits=1)
| metric | knn | ari | epoch_time(s) | |||
|---|---|---|---|---|---|---|
| mean±SD | mean | SD | mean | SD | mean | SD |
| 0 | 0.84 | 0.00 | 0.76 | 0.00 | 0.09 | 0.00 |
Sinkhorn on CPU#
Even on CPU (settings.device="cpu") the computation time benefits from batched tensor operations and better utilization of large number of CPU threads (settings.n_threads) compared to the slow training using EMD2 (entropic_reg=0).
[5]:
ggml_ot.settings.device = "cpu"
ggml_ot.settings.n_threads = 64
_, scores["sinkhorn_cpu"] = dataset.train_test(entropic_reg=5.0, n_splits=1)
| metric | knn | ari | epoch_time(s) | |||
|---|---|---|---|---|---|---|
| mean±SD | mean | SD | mean | SD | mean | SD |
| 0 | 0.84 | 0.00 | 0.70 | 0.00 | 1.09 | 0.00 |
Exact EMD2 on GPU and CPU#
For contrast, we run the exact EMD2 solver (entropic_reg=0.0, the default) on both devices. EMD2 is computed by the network-simplex LP solver shipped with POT, which is not batched and runs on the CPU. Setting settings.device="cuda" therefore only moves tensors to the GPU for the triplet loss, the OT solves themselves stay on the CPU and train() emits a corresponding notification.
[6]:
ggml_ot.settings.device = "cuda"
_, scores["emd2_gpu"] = dataset.train_test(entropic_reg=0.0, n_splits=1)
[ggml_ot] Exact EMD2 solver is CPU-bound, to leverage CUDA use Sinkhorn solver (pass parameter entropic_reg > 0).
| metric | knn | ari | epoch_time(s) | |||
|---|---|---|---|---|---|---|
| mean±SD | mean | SD | mean | SD | mean | SD |
| 0 | 0.84 | 0.00 | 0.65 | 0.00 | 144.35 | 0.00 |
[7]:
ggml_ot.settings.device = "cpu"
# entropic_reg=0.0 (emd2) is default
_, scores["emd2_cpu"] = dataset.train_test(n_splits=1)
| metric | knn | ari | epoch_time(s) | |||
|---|---|---|---|---|---|---|
| mean±SD | mean | SD | mean | SD | mean | SD |
| 0 | 0.84 | 0.00 | 0.75 | 0.00 | 127.38 | 0.00 |
As expected, the non-batched EMD2 cannot leverage GPU speed-up and massive parallelization.
Performance comparison#
train_test() reports the mean epoch time alongside the usual classification and clustering metrics. We sort by epoch time and show the four setups from fastest to slowest.
[8]:
scores_df = (
pd.concat([scores[setup] for setup in scores], ignore_index=True)
.set_index(pd.Index(scores.keys(), name="setup"))
.sort_values(("epoch_time(s)", "mean"))
)
ggml_ot.pl.table(
scores_df,
style_performance=True,
title="Sinkhorn vs. EMD2 on GPU and CPU",
)
| metric | knn | ari | epoch_time(s) | |||
|---|---|---|---|---|---|---|
| mean±SD | mean | SD | mean | SD | mean | SD |
| setup | ||||||
| sinkhorn_gpu | 0.84 | 0.00 | 0.76 | 0.00 | 0.09 | 0.00 |
| sinkhorn_cpu | 0.84 | 0.00 | 0.70 | 0.00 | 1.09 | 0.00 |
| emd2_cpu | 0.84 | 0.00 | 0.75 | 0.00 | 127.38 | 0.00 |
| emd2_gpu | 0.84 | 0.00 | 0.65 | 0.00 | 144.35 | 0.00 |
Takeaways:
To leverage GPU speed-up the Sinkhorn solver must be used with
entropic_reg > 0normalize()before tuningentropic_regmakes regularization strengths transferable across datasets
EMD2 solver is not batched and computations are CPU-bound regardless of
settings.device
See the fast approximations tutorial for other computational speed-ups for both, EMD2 and Sinkhorn.
Note
To improve training time also consider other speed-up strategies like PCA embeddings, cell-type grouping, and cell subsampling, see Tutorial on Computational Speed-ups.