Morphological Prototyping for Unsupervised Slide Representation Learning in Computational Pathology

Andrew H. SongRichard J. ChenTong DingDrew F. K. WilliamsonGuillaume JaumeFaisal Mahmood

article2024CVPR124 citations

Proposes an unsupervised Gaussian mixture model framework that condenses whole-slide images into compact morphological prototypes, matching or outperforming supervised multiple instance learning baselines across subtyping and survival prediction benchmarks while enabling interpretable slide-level analysis.

Listen

Analyzing gigapixel whole-slide pathology images is central to modern clinical diagnosis and cancer prognosis. Most current computational pathology workflows rely on weakly supervised multiple instance learning, which trains models directly on specific clinical outcomes. While effective for simple diagnostic tasks that depend on finding small focal abnormalities, these weakly supervised models struggle to generalize across diverse clinical questions and fail to comprehensively capture complex tissue microenvironments, such as the proportions, mixtures, and spatial heterogeneity of various cell populations, especially when labeled training data is scarce.

The article introduces and evaluates PANTHER, an unsupervised slide representation framework that summarizes massive histology images into compact, general-purpose profiles without requiring clinical labels during feature extraction. The primary objective is to demonstrate that summarizing tissue images into a concise set of morphological prototypes provides task-agnostic representations that match or exceed the accuracy of supervised models while offering direct clinical interpretability.

The method leverages the structural redundancy of human tissue by using a Gaussian mixture model to represent each whole-slide image. Large images are partitioned into tens of thousands of smaller patches, each processed by a pretrained vision encoder. PANTHER estimates mixture parameters using an expectation-maximization procedure, mapping each patch softly to a dictionary of shared morphological prototypes. Rather than averaging these features, the framework concatenates the estimated mixture weights, means, and variances across all prototypes into a single fixed-length slide profile, which can then feed lightweight linear or multilayer perceptron predictors.

The evaluation across 13 datasets encompassing four cancer subtyping tasks and nine survival outcome benchmarks produced three central findings. First, PANTHER coupled with a lightweight multilayer perceptron consistently matched or outperformed leading supervised multiple instance learning baselines across both subtyping and patient survival prediction. Second, PANTHER substantially outperformed existing unsupervised set-representation baselines, demonstrating that explicitly preserving both deep visual features and prototype extent (cardinality) through concatenation is critical for complex clinical prediction. Third, qualitative and quantitative validation confirmed that the learned prototypes accurately map distinct tissue structures, such as separating adenocarcinoma from squamous cell carcinoma patterns and isolating tumor-infiltrating immune cells.

These findings indicate that general-purpose, unsupervised slide embeddings can reduce dependence on costly task-specific model training and extensive manual annotations. By decoupling slide representation learning from clinical outcome labels, healthcare organizations and developers can lower computational overhead, reuse standardized slide summaries across multiple diagnostic and prognostic tasks, and mitigate risks associated with overfitting to small datasets. Furthermore, the ability to trace predictions back to specific morphological prototypes and generate visual assignment maps provides transparent interpretability that supports clinical auditability.

Organizations developing computational pathology pipelines should consider adopting prototype-based unsupervised aggregation to build reusable slide feature repositories. Next steps should focus on running data-driven pilots to determine the optimal number of prototypes dynamically across diverse cancer types and testing representations on smaller, rarer disease cohorts. Readers should note that the current evaluation fixed the number of prototypes to sixteen across all cancer types, which may lead to slight over- or under-clustering in specific tissue contexts, though overall confidence in the benchmarked results remains strong.

Emory University Harvard University Mass General Brigham rapid prototyping representation learning supervised machine learning unsupervised slide representation learning morphological prototyping computational pathology

arXiv: 2405.11643

Paper: Data-efficient and weakly supervised computational pathology on whole-slide images, Ming Y. Lu et al. (2020). Introduces the CLAM framework and clustering-constrained attention mechanisms for whole-slide histopathology, establishing the foundational weakly supervised baseline that PANTHER seeks to replace with unsupervised morphological prototypes.
Paper: Attention-based Deep Multiple Instance Learning, Maximilian Ilse et al. (2018). Presents attention-based multiple instance learning for pathology bags, defining the primary weak-supervision paradigm whose task-specific limitations motivate PANTHER's unsupervised prototype formulation.
Paper: Benchmarking Self-Supervised Learning on Diverse Pathology Datasets, Mingu Kang et al. (2023). Provides a comprehensive benchmark of self-supervised learning methods across diverse pathology datasets, offering essential background on learning patch-level representations without manual annotations.
Paper: An Analysis of Single-Layer Networks in Unsupervised Feature Learning, Adam Coates et al. (2011). Demonstrates the efficacy of unsupervised Gaussian mixture and clustering models for local patch feature aggregation, which directly underpins PANTHER's mixture-based morphological prototyping.

Paper: ViLa-MIL: Dual-scale Vision-Language Multiple Instance Learning for Whole Slide Image Classification, Jiangbo Shi et al. (2024). Builds upon prototype-based patch grouping in whole slide images by integrating dual-scale vision-language decoders and text prompts for few-shot pathology classification.

Cover for Morphological Prototyping for Unsupervised Slide Representation Learning in Computational Pathology

Abstract

Representation learning of pathology whole-slide images (WSIs) has been has primarily relied on weak supervision with Multiple Instance Learning (MIL). However, the slide representations resulting from this approach are highly tailored to specific clinical tasks, which limits their expressivity and generalization, particularly in scenarios with limited data. Instead, we hypothesize that morphological redundancy in tissue can be leveraged to build a task-agnostic slide representation in an unsupervised fashion. To this end, we introduce PANTHER, a prototype-based approach rooted in the Gaussian mixture model that summarizes the set of WSI patches into a much smaller set of morphological prototypes. Specifically, each patch is assumed to have been generated from a mixture distribution, where each mixture component represents a morphological exemplar. Utilizing the estimated mixture parameters, we then construct a compact slide representation that can be readily used for a wide range of downstream tasks. By performing an extensive evaluation of PANTHER on subtyping and survival tasks using 13 datasets, we show that 1) PANTHER outperforms or is on par with supervised MIL baselines and 2) the analysis of morphological prototypes brings new qualitative and quantitative insights into model interpretability. The code is available at https://github.com/mahmoodlab/Panther.

Topics

rapid prototyping representation learning supervised machine learning unsupervised learning method

Keywords

unsupervised slide representation learning morphological prototyping computational pathology slide image classification prototypical assignment map histopathology images contrastive learning self-supervised learning

1. Introduction
2. Related work
2.1. Multiple instance learning in CPath
2.2. Quantification of distance between sets
2.3. Prototype-based set representation
3. Methods
3.1. Prototype-based aggregation
3.1.1 Connection to optimal transport
3.2. Downstream evaluation
3.3. Interpretability
4. Experiments
4.1. Datasets
4.2. Baselines
4.3. Implementation
5. Results
5.1. Subtyping and survival prediction
5.2. Interpretability
5.3. Further ablations
6. Conclusion and limitations
References

Knowls

Knowl 1 — PANTHER represents a whole-slide image as Gaussian morphological prototypes
model/method
PANTHER converts a whole-slide image (WSI) into a compact, task-agnostic representation by modeling the distribution of its patch embeddings with a Gaussian mixture model (GMM). For subject $j$ , non-overlapping image patches $x_n^j\in\mathbb{R}^{W\times H\times 3}$ are encoded by a frozen histopathology feature extractor $f_{\mathrm{enc}}$ into embeddings $z_n^j=f_{\mathrm{enc}}(x_n^j)\in\mathbb{R}^d$ , where $n=1,\ldots,N_j$ indexes the patches. A cohort-level set of $C$ prototype centroids $H=\{h_1,\ldots,h_C\}$ is obtained by K-means clustering, with $C\ll N_j$ .

For each WSI, PANTHER fits separate mixture parameters consisting of a component probability $\hat{\pi}_c^j$ , mean $\hat{\mu}_c^j\in\mathbb{R}^d$ , and diagonal covariance $\hat{\Sigma}_c^j\in\mathbb{R}^{d\times d}$ for each prototype $c$ . The slide embedding concatenates the parameters of all components rather than averaging them:
$z_{\mathrm{WSI}}^j=[\hat{\pi}_1^j,\hat{\mu}_1^j,\hat{\Sigma}_1^j,\ldots,\hat{\pi}_C^j,\hat{\mu}_C^j,\hat{\Sigma}_C^j]\in\mathbb{R}^{C(1+2d)}.$
The mixture probability explicitly records the extent or cardinality of a morphological pattern in the slide, while the component mean and covariance describe that pattern and its within-slide variation. The shared prototype initialization makes components comparable across WSIs, but all mixture parameters are estimated independently for each WSI.
Knowl 2 — GMM fitting provides soft patch-to-prototype assignments and the slide representation
model/method
For a WSI with patch embeddings $Z^j=\{z_1^j,\ldots,z_{N_j}^j\}$ , PANTHER assumes each embedding is generated by one of $C$ Gaussian components. The component probability is $\pi_c^j\ge 0$ with $\sum_{c=1}^C\pi_c^j=1$ , the mean is $\mu_c^j\in\mathbb{R}^d$ , and the covariance $\Sigma_c^j$ is diagonal:
$p(z_n^j;\theta^j)=\sum_{c=1}^{C}\pi_c^j\,\mathcal{N}(z_n^j;\mu_c^j,\Sigma_c^j),\qquad \theta^j=\{\pi_c^j,\mu_c^j,\Sigma_c^j\}_{c=1}^{C}.$
The parameters maximize the per-slide log-likelihood $\sum_{n=1}^{N_j}\log p(z_n^j;\theta^j)$ using expectation-maximization (EM). At iteration $t+1$ , the posterior assignment probability of patch $n$ to component $c$ is
$q_{n,c}^{j,(t+1)}=\frac{\pi_c^{j,(t)}\mathcal{N}(z_n^j;\mu_c^{j,(t)},\Sigma_c^{j,(t)})}{\sum_{r=1}^{C}\pi_r^{j,(t)}\mathcal{N}(z_n^j;\mu_r^{j,(t)},\Sigma_r^{j,(t)})}.$
The M-step updates the mixture probability, mean, and diagonal covariance using these soft assignments:
$\pi_c^{j,(t+1)}=\frac{1}{N_j}\sum_{n=1}^{N_j}q_{n,c}^{j,(t+1)},\qquad \mu_c^{j,(t+1)}=\frac{\sum_{n=1}^{N_j}q_{n,c}^{j,(t+1)}z_n^j}{\sum_{n=1}^{N_j}q_{n,c}^{j,(t+1)}},$ $\Sigma_c^{j,(t+1)}=\frac{\sum_{n=1}^{N_j}q_{n,c}^{j,(t+1)}(z_n^j-\mu_c^{j,(t+1)})^2}{\sum_{n=1}^{N_j}q_{n,c}^{j,(t+1)}},$
where the squared operation is elementwise because the covariance is diagonal. Initialization uses $\pi_c^{j,(0)}=1/C$ , $\mu_c^{j,(0)}=h_c$ , and $\Sigma_c^{j,(0)}=I_d$ . The resulting soft assignments allow every patch to contribute to every prototype, weighted by its posterior probability. In the reported implementation, one EM step was sufficient for convergence.
Knowl 3 — Prototype-wise nonlinear prediction uses a structured MLP
model/method
PANTHER can feed the concatenated prototype parameters to a linear predictor or to a structured multilayer perceptron (MLP). Let $z_{\mathrm{WSI},c}$ denote the parameter block $[\hat{\pi}_c,\hat{\mu}_c,\hat{\Sigma}_c]$ for prototype $c$ , with $c=1,\ldots,C$ . Instead of applying one MLP to the entire concatenated vector, the structured predictor first applies a separate function $g_c^{\mathrm{indiv}}$ to each prototype block and then applies a prediction function $g^{\mathrm{pred}}$ :
$z'_{\mathrm{WSI}}=[g_1^{\mathrm{indiv}}(z_{\mathrm{WSI},1}),\ldots,g_C^{\mathrm{indiv}}(z_{\mathrm{WSI},C})],\qquad g(z_{\mathrm{WSI}})=g^{\mathrm{pred}}(z'_{\mathrm{WSI}}).$
Each $g_c^{\mathrm{indiv}}$ and $g^{\mathrm{pred}}$ can be an identity map, a linear layer, or an MLP. This architecture preserves access to each morphological component while allowing prototype-specific nonlinear transformations. It is feasible because PANTHER compresses each variable-length patch set into a fixed number of prototype blocks; a conventional patch-level MIL model cannot readily learn one function per patch because the number of patches varies and patch ordering is permutation-invariant.
Knowl 4 — PANTHER is a nonuniform prototype aggregation related to optimal transport
theoretical result
PANTHER can be interpreted as matching the empirical distribution of patch embeddings to a distribution supported on prototype centroids. For WSI $j$ , let $\delta_u$ denote a point mass at embedding $u$ , let $a_n^j$ be the mass assigned to patch embedding $z_n^j$ , and let $h_c$ be the shared centroid for prototype $c$ . The two distributions are
$\hat p^j=\sum_{n=1}^{N_j}a_n^j\delta_{z_n^j},\qquad \hat q^j=\sum_{c=1}^{C}\pi_c^j\delta_{h_c},$
with $a_n^j=1/N_j$ , $\sum_n a_n^j=1$ , and $\sum_c\pi_c^j=1$ . Optimal-transport aggregation typically fixes both the patch masses and prototype masses to uniform values, including $\pi_c^j=1/C$ . PANTHER also uses uniform patch masses but estimates the prototype masses $\pi_c^j$ from the WSI-specific GMM. Consequently, unlike uniform optimal transport, PANTHER retains how prevalent each morphological prototype is in each slide. The paper notes that the optimal-transport solution with uniform prototype weights can be viewed as a special case of the GMM formulation.
Knowl 5 — Prototype assignment maps make the learned representation spatially interpretable
model/method
PANTHER produces a visual explanation of a WSI by assigning each patch to the prototype with the largest posterior probability. For patch embedding $z_n^j$ , the hard visualization label is
$c_n^j=\arg\max_{c\in\{1,\ldots,C\}}q(c_n^j=c\mid z_n^j),$
where $q(c_n^j=c\mid z_n^j)$ is the final GMM posterior assignment probability. The labels $c_n^j$ are placed at the corresponding patch locations to form a prototypical assignment map. The estimated mixture probability $\hat\pi_c^j$ quantifies the proportion or extent of prototype $c$ in the WSI. For a selected prototype $c_0$ , the unthresholded probability $q(c_n^j=c_0\mid z_n^j)$ gives a graded map of how morphologically similar each patch is to that prototype. Thus, PANTHER supports both slide-level quantification of prototype prevalence and spatial visualization of where each learned morphology occurs.
Knowl 6 — Experimental evaluation covers four subtyping tasks and nine survival evaluations
experimental setup
PANTHER was evaluated with the same UNI histopathology features as the baselines. UNI is a ViT-L/16 DINOv2 encoder pretrained on approximately $10^8$ patches from $10^6$ whole-slide images. WSIs were processed at $20\times$ magnification, corresponding to $0.5\,\mu\mathrm{m}$ /pixel, using non-overlapping $256\times256$ -pixel patches; all patches were retained without sampling. Prototype centroids were obtained by K-means on patches pooled from all training slides in the cohort. The reported comparisons used $C=16$ prototypes for PANTHER and the prototype-based baselines.

The four subtyping evaluations were EBRAINS fine subtyping with 32 classes, EBRAINS coarse subtyping with 12 classes, non-small-cell lung carcinoma (NSCLC) subtyping with two classes using TCGA and CPTAC, and PANDA prostate ISUP grading with six classes. Balanced accuracy and weighted F1 were used for EBRAINS and NSCLC, and Cohen's $\kappa$ was used for ISUP grading.

Disease-specific survival was evaluated for BRCA, CRC, BLCA, UCEC, KIRC, and LUAD. TCGA experiments used five-fold site-stratified cross-validation. KIRC models trained on TCGA were additionally tested on CPTAC, and LUAD models were additionally tested on CPTAC and NLST, yielding nine survival test settings. Survival performance was measured by the concordance index. Baselines included unsupervised DeepSets, ProtoCounts, H2T, and optimal transport representations, as well as supervised ABMIL, TransMIL, DSMIL, AttnMISL, and ILRA.
Knowl 7 — PANTHER achieves competitive subtyping results, with full parameter concatenation outperforming reduced variants
data/table
The following results compare subtyping metrics using UNI features. PANTHER variants and the prototype-based baselines use $C=16$ prototypes. The full PANTHER representation concatenates every mixture probability, mean, and covariance; the weighted-average variant averages means and covariances using mixture probabilities; Top and Bottom retain only the component with the largest or smallest mixture probability. The results show that PANTHER $_{\mathrm{All}}+$ MLP is competitive with supervised MIL and generally stronger than unsupervised baselines, while retaining all prototype blocks is substantially better than retaining only one component.
Could not parse LaTeX table
The full nonlinear representation obtains the strongest unsupervised results in every listed subtyping setting and is close to the best supervised model. The comparison of PANTHER $_{\mathrm{All}}$ with PANTHER $_{\mathrm{WA}}$ , Top, and Bottom supports the contribution's claim that both prototype identity and prototype extent should be preserved without averaging away the component-specific information.
Knowl 8 — PANTHER improves or matches baselines across diverse disease-specific survival tasks
data/table
The following values are concordance indices for disease-specific survival, with standard deviations over five runs. All representations and prototypes were trained on TCGA; CPTAC and NLST columns are external tests. PANTHER $_{\mathrm{All}}+$ MLP is the strongest or near-strongest method across the nine test settings, especially on external KIRC and LUAD evaluations, demonstrating that the unsupervised representation transfers to prognostic prediction.
Could not parse LaTeX table
The authors report that PANTHER $_{\mathrm{All}}+$ MLP outperforms most unsupervised baselines and is on par with or better than supervised MIL overall. The nonlinear head consistently improves over the linear head. The results support the value of jointly encoding deep prototype descriptors and their slide-specific cardinalities.
Knowl 9 — Ablations show that preserving all morphological components and their identities is essential
empirical result
PANTHER variants were compared by retaining all mixture parameters, averaging prototype means and covariances using their mixture probabilities, or retaining only the most prevalent or least prevalent component. The full concatenated representation, PANTHER $_{\mathrm{All}}$ , consistently outperformed the weighted-average representation, PANTHER $_{\mathrm{WA}}$ , on the reported subtyping and survival evaluations. The Top and Bottom variants were generally poor, showing that a single dominant or rare morphology does not adequately summarize a heterogeneous WSI.

The authors attribute the advantage of concatenation to two properties: each prototype remains directly addressable by the predictor, and the representation retains both morphology and prevalence. ProtoCounts retains prevalence without deep prototype descriptors and performed poorly, whereas simple averaging methods such as DeepSets and PANTHER $_{\mathrm{WA}}$ can blur discriminative component-specific information. The structured MLP further improves prediction over a linear head, indicating that prototype-specific nonlinear interactions provide additional predictive capacity. Supplementary experiments reported robustness to the choice of prototype count, survival loss, and feature encoder.
Knowl 10 — Learned prototypes correspond to clinically meaningful tissue morphologies
empirical result
Visual inspection by a board-certified pathologist found that PANTHER prototypes represented distinct phenotypes of tumor, tumor-associated stroma, immune-cell populations, and normal tissue. In NSCLC, prototypes C2 and C15 corresponded to adenocarcinoma morphology, while prototype C12 corresponded to squamous cell carcinoma morphology. Their estimated mixture probabilities were concentrated almost exclusively in LUAD and LUSC slides, respectively, indicating that prototype prevalence can capture disease-associated morphological composition.

For colorectal cancer, the distribution of learned prototype patterns showed strong concordance with existing CRC-100K tissue-type annotations. These results demonstrate that the GMM assignments are not only predictive features: their spatial maps and prevalence values provide post-hoc quantitative and visual descriptions of tissue heterogeneity.
Knowl 11 — Fixed prototype count is a stated limitation of PANTHER
limitation
All reported tasks used $C=16$ mixture components. A single fixed value may over-cluster some cancer types and under-cluster others, so the learned components may not provide an equally appropriate morphological vocabulary for every cohort. The authors identify more expressive mixture models, data-driven selection of the number of prototypes, and evaluation on rare cancer cohorts with small sample sizes as directions for future work.

Coverage note — Additional supplementary implementation, loss-function, encoder, and prototype-count ablations were omitted because they provide supporting rather than load-bearing contributions beyond the ten highest-significance knowls.

References

1.Shekoofeh Azizi, Laura Culp, Jan Freyberg, Basil Mustafa, Sebastien Baur, Simon Kornblith, Ting Chen, Nenad Tomasev, Jovana Mitrovic, Patricia Strachan, et al. Robust and data-efficient generalization of self-supervised machine learning for diagnostic imaging. Nature Biomedical Engineering, pages 1–24, 2023. 1
2.Saurav Basu, Soheil Kolouri, and Gustavo K Rohde. Detecting and visualizing cell phenotype differences from microscopy images using transport-based morphometry. Proceedings of the National Academy of Sciences, 111(9):3448–3453, 2014. 2
3.Andrew H Beck, Ankur R Sangoi, Samuel Leung, Robert J Marinelli, Torsten O Nielsen, Marc J Van De Vijver, Robert B West, Matt Van De Rijn, and Daphne Koller. Systematic analysis of breast cancer morphology uncovers stromal features associated with survival. Science Translational Medicine, 3(108):108ra113–108ra113, 2011. 2
4.Babak Ehteshami Bejnordi, Mitko Veta, Paul Johannes Van Diest, Bram Van Ginneken, Nico Karssemeijer, Geert Litjens, Jeroen AWM Van Der Laak, Meyke Hermsen, Quirine F Manson, Maschenka Balkenhol, et al. Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer. JAMA, 318(22):2199–2210, 2017. 2
5.Gianpaolo Bontempo, Angelo Porrello, Federico Bolelli, Simone Calderara, and Elisa Ficarra. Das-mil: Distilling across scales for mil classification of histological wsis. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 248–258. Springer, 2023. 2
6.James D Brierley, Mary K Gospodarowicz, and Christian Wittekind. TNM classification of malignant tumours. John Wiley & Sons, 2017. 2
7.Christian Bueno and Alan Hylton. On the representation power of set pooling networks. Advances in Neural Information Processing Systems, 34:17170–17182, 2021. 2
8.Wouter Bulten, Kimmo Kartasalo, Po-Hsuan Cameron Chen, Peter Strom, Hans Pinckaers, Kunal Nagpal, Yuannan Cai, David F Steiner, Hester van Boven, Robert Vink, et al. Artificial intelligence for diagnosis and gleason grading of prostate cancer: the panda challenge. Nature Medicine, 28(1):154–163, 2022. 5, 2
9.Wouter Bulten, Hans Pinckaers, Hester van Boven, Robert Vink, Thomas de Bel, Bram van Ginneken, Jeroen van der Laak, Christina Hulsbergen-van de Kaa, and Geert Litjens. Automated deep-learning system for gleason grading of prostate cancer using biopsies: a diagnostic study. The Lancet Oncology, 21(2):233–241, 2020. 5, 2
10.Charlotte Bunne, Stefan G Stark, Gabriele Gut, Jacobo Sarabia Del Castillo, Mitch Levesque, Kjong-Van Lehmann, Lucas Pelkmans, Andreas Krause, and Gunnar Ratsch. Learning single-cell perturbation responses using neural optimal transport. Nature methods, pages 1–10, 2023. 2
11.Dmitrii Bychkov, Nina Linder, Riku Turkki, Stig Nordling, Panu E Kovanen, Clare Verrill, Margarita Walliander, Mikael Lundin, Caj Haglund, and Johan Lundin. Deep learning based tissue analysis predicts outcome in colorectal cancer. Scientific Reports, 8(1), 2018. 2
12.Juan C Caicedo, Angel Cruz, and Fabio A Gonzalez. Histopathology image classification using bag of features and kernel functions. In Artificial Intelligence in Medicine: 12th Conference on Artificial Intelligence in Medicine, AIME 2009, Verona, Italy, July 18-22, 2009. Proceedings 12, pages 126–135. Springer, 2009. 3
13.Gabriele Campanella, Matthew G Hanna, Luke Geneslaw, Allen Miraflor, Vitor Werneck Krauss Silva, Klaus J Busam, Edi Brogi, Victor E Reuter, David S Klimstra, and Thomas J Fuchs. Clinical-grade computational pathology using weakly supervised deep learning on whole slide images. Nature medicine, 25(8):1301–1309, 2019. 1, 2
14.Iain Carmichael, Andrew H Song, Richard J Chen, Drew FK Williamson, Tiffany Y Chen, and Faisal Mahmood. Incorporating intratumoral heterogeneity into weakly-supervised deep learning models via variance pooling. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 387–397. Springer, 2022. 5
15.Lyndon Chan, Mahdi S Hosseini, Corwyn Rowsell, Konstantinos N Plataniotis, and Savvas Damaskinos. Histosegnet: Semantic segmentation of histological tissue type in whole slide images. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 10662–10671, 2019. 2
16.Chaofan Chen, Oscar Li, Daniel Tao, Alina Barnett, Cynthia Rudin, and Jonathan K Su. This looks like that: deep learning for interpretable image recognition. Advances in neural information processing systems, 32, 2019. 3
17.Richard J Chen, Chengkuan Chen, Yicong Li, Tiffany Y Chen, Andrew D Trister, Rahul G Krishnan, and Faisal Mahmood. Scaling vision transformers to gigapixel images via hierarchical self-supervised learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16144–16155, 2022. 1, 2
18.Richard J. Chen, Tong Ding, Ming Y. Lu, Drew F. K. Williamson, Guillaume Jaume, Andrew H. Song, Bowen Chen, Andrew Zhang, Daniel Shao, Muhammad Shaban, Mane Williams, Lukas Oldenburg, Luca L. Weishaupt, Judy J. Wang, Anurag Vaidya, Long Phi Le, Georg Gerber, Sharifa Sahai, Walt Williams, and Faisal Mahmood. Towards a general-purpose foundation model for computational pathology. Nature Medicine, 2024. 1, 3, 5, 6, 7, 2
19.Yixin Chen and James Z Wang. Image categorization by learning and reasoning with regions. The Journal of Machine Learning Research, 5:913–939, 2004. 3
20.Zixiang Chen, Yihe Deng, Yue Wu, Quanquan Gu, and Yuanzhi Li. Towards understanding the mixture-of-experts layer in deep learning. Advances in neural information processing systems, 35:23049–23062, 2022. 2
21.Ozan Ciga, Tony Xu, and Anne Louise Martel. Self supervised contrastive learning for digital histopathology. Machine Learning with Applications, 7:100198, 2022. 1
22.Nicolas Coudray, Paolo Santiago Ocampo, Theodore Sakellaropoulos, Navneet Narula, Matija Snuderl, David Fenyo, Andre L Moreira, Narges Razavian, and Aristotelis Tsirigos. Classification and mutation prediction from non–small cell lung cancer histopathology images using deep learning. Nature medicine, 24(10):1559–1567, 2018. 2
23.Angel Cruz-Roa, Juan C Caicedo, and Fabio A Gonzalez. Visual pattern mining in histology image collections using bag of features. Artificial intelligence in medicine, 52(2):91–106, 2011. 3
24.Yufei Cui, Ziquan Liu, Yixin Chen, Yuchen Lu, Xinyue Yu, Xue Liu, Tei-Wei Kuo, Miguel R. D. Rodrigues, Chun Jason Xue, and Antoni B. Chan. Retrieval-augmented multiple instance learning. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. 2
25.Marco Cuturi. Sinkhorn distances: Lightspeed computation of optimal transport. Advances in neural information processing systems, 26, 2013. 2, 4, 5
26.Dan dan Guo, Long Tian, Minghe Zhang, Mingyuan Zhou, and Hongyuan Zha. Learning prototype-oriented set representations for meta-learning. In International Conference on Learning Representations, 2022. 2, 3
27.Arthur P Dempster, Nan M Laird, and Donald B Rubin. Maximum likelihood from incomplete data via the em algorithm. Journal of the royal statistical society: series B (methodological), 39(1):1–22, 1977. 2, 3, 1
28.Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations, 2021. 5
29.M Murat Dundar, Sunil Badve, Vikas C Raykar, Rohit K Jain, Olcay Sertel, and Metin N Gurcan. A multiple instance learning approach toward optimal classification of pathology slides. In 2010 20th International Conference on Pattern Recognition, pages 2732–2735. IEEE, 2010. 1
30.Kianoush Falahkheirkhah, Alex Xijie Lu, David Alvarez-Melis, and Grace Huynh. Domain adaptation using optimal transport for invariant learning using histopathology datasets. In Medical Imaging with Deep Learning, 2023. 2
31.Alexandre Filiot, Ridouane Ghermi, Antoine Olivier, Paul Jacob, Lucas Fidon, Alice Mac Kain, Charlie Saillard, and Jean-Baptiste Schiratti. Scaling self-supervised learning for histopathology with masked image modeling. medRxiv, pages 2023–07, 2023. 1
32.Gemma Gatta, Riccardo Capocaccia, Laura Botta, Sandra Mallone, Roberta De Angelis, Eva Ardanaz, Harry Comber, Nadya Dimitrova, Maarit K Leinonen, Sabine Siesling, et al. Burden and centralised treatment in europe of rare tumours: results of rarecarenet—a population-based study. The Lancet Oncology, 18(8):1022–1039, 2017. 5
33.Noriaki Hashimoto, Daisuke Fukushima, Ryoichi Koga, Yusuke Takagi, Kaho Ko, Kei Kohno, Masato Nakaguro, Shigeo Nakamura, Hidekata Hontani, and Ichiro Takeuchi. Multi-scale domain-adversarial multiple-instance cnn for cancer subtype classification with unannotated histopathological images. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 3852–3861, 2020. 2
34.Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. 5
35.Andreas Heindl, Sidra Nawaz, and Yinyin Yuan. Mapping spatial heterogeneity in the tumor microenvironment: a new era for digital pathology. Laboratory investigation, 95(4):377–384, 2015. 2
36.Le Hou, Dimitris Samaras, Tahsin M Kurc, Yi Gao, James E Davis, and Joel H Saltz. Patch-based convolutional neural network for whole slide tissue image classification. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2424–2433, 2016. 3
37.Wentai Hou, Lequan Yu, Chengxuan Lin, Helong Huang, Rongshan Yu, Jing Qin, and Liansheng Wang. Hˆ 2-mil: exploring hierarchical representation with heterogeneous multiple instance learning for whole slide image analysis. In Proceedings of the AAAI conference on artificial intelligence, volume 36, pages 933–941, 2022. 2
38.Frederick M Howard, James Dolezal, Sara Kochanny, Jefree Schulte, Heather Chen, Lara Heij, Dezheng Huo, Rita Nanda, Olufunmilayo I Olopade, Jakob N Kather, et al. The impact of site-specific digital histology signatures on deep learning model accuracy and bias. Nature communications, 12(1):4423, 2021. 2
39.Zhi Huang, Federico Bianchi, Mert Yuksekgonul, Thomas J Montine, and James Zou. A visual–language foundation model for pathology image analysis using medical twitter. Nature medicine, 29(9):2307–2316, 2023. 1
40.Maximilian Ilse, Jakub Tomczak, and Max Welling. Attention-based deep multiple instance learning. In International conference on machine learning, pages 2127–2136. PMLR, 2018. 1, 2, 5, 6, 7
41.Guillaume Jaume, Lukas Oldenburg, Anurag J Vaidya, Richard J Chen, Drew FK Williamson, Thomas Peeters, Andrew H Song, and Faisal Mahmood. Transcriptomics-guided slide representation learning in computational pathology. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024. 1
42.Guillaume Jaume, Anurag Vaidya, Richard Chen, Drew Williamson, Paul Liang, and Faisal Mahmood. Modeling dense multimodal interactions between biological pathways and histology for survival prediction. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024. 5
43.Syed Ashar Javed, Dinkar Juyal, Harshith Padigela, Amaro Taylor-Weiner, Limin Yu, and Aaditya Prakash. Additive mil: intrinsically interpretable multiple instance learning for pathology. Advances in Neural Information Processing Systems, 35:20689–20702, 2022. 2
44.Cheng Jiang, Xinhai Hou, Akhil Kondepudi, Asadur Chowdury, Christian W Freudiger, Daniel A Orringer, Honglak Lee, and Todd C Hollon. Hierarchical discriminative learning improves visual representations of biomedical microscopy. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19798–19808, 2023. 1
45.Shivam Kalra, Hamid R Tizhoosh, Charles Choi, Sultaan Shah, Phedias Diamandis, Clinton JV Campbell, and Liron Pantanowitz. Yottixel–an image search engine for large archives of histopathology whole slide images. Medical Image Analysis, 65:101757, 2020. 3
46.Mingu Kang, Heon Song, Seonwook Park, Donggeun Yoo, and Sergio Pereira. Benchmarking self-supervised learning on diverse pathology datasets. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3344–3354, 2023. 1
47.Jakob Nikolas Kather, Johannes Krisam, Pornpimol Charoentong, Tom Luedde, Esther Herpel, Cleo-Aron Weis, Timo Gaiser, Alexander Marx, Nektarios A Valous, Dyke Ferber, et al. Predicting survival from colorectal cancer histology slides using deep learning: A retrospective multicenter study. PLoS Medicine, 16(1), 2019. 2, 7, 3
48.Jared L Katzman, Uri Shaham, Alexander Cloninger, Jonathan Bates, Tingting Jiang, and Yuval Kluger. Deepsurv: personalized treatment recommender system using a cox proportional hazards deep neural network. BMC medical research methodology, 18(1):1–12, 2018. 2
49.Minyoung Kim. Differentiable expectation-maximization for set representation learning. In International Conference on Learning Representations, 2022. 2, 3, 4, 1
50.Soheil Kolouri, Se Rim Park, Matthew Thorpe, Dejan Slepcev, and Gustavo K. Rohde. Optimal mass transport: Signal processing and machine-learning applications. IEEE Signal Processing Magazine, 34(4):43–59, 2017. 2
51.Daisuke Komura, Akihiro Kawabe, Keisuke Fukuta, Kyohei Sano, Toshikazu Umezaki, Hirotomo Koda, Ryohei Suzuki, Ken Tominaga, Mieko Ochi, Hiroki Konishi, et al. Universal encoding of pan-cancer histology by deep texture representations. Cell Reports, 38(9), 2022. 3
52.Tristan Lazard, Marvin Lerousseau, Etienne Decenciere, and Thomas Walter. Giga-ssl: Self-supervised learning for gigapixel images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4304–4313, 2023. 1
53.Svetlana Lazebnik, Cordelia Schmid, and Jean Ponce. A sparse texture representation using local affine regions. IEEE transactions on pattern analysis and machine intelligence, 27(8):1265–1278, 2005. 3
54.Juho Lee, Yoonho Lee, Jungtaek Kim, Adam Kosiorek, Seungjin Choi, and Yee Whye Teh. Set transformer: A framework for attention-based permutation-invariant neural networks. In International conference on machine learning, pages 3744–3753. PMLR, 2019. 3
55.Bin Li, Yin Li, and Kevin W Eliceiri. Dual-stream multiple instance learning network for whole slide image classification with self-supervised contrastive learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14318–14328, 2021. 1, 2, 6, 7, 5
56.Tiancheng Lin, Zhimiao Yu, Hongyu Hu, Yi Xu, and Chang-Wen Chen. Interventional bag multi-instance learning on whole-slide pathological images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19830–19839, 2023. 1
57.Jianfang Liu, Tara Lichtenberg, Katherine A Hoadley, Laila M Poisson, Alexander J Lazar, Andrew D Cherniack, Albert J Kovatich, Christopher C Benz, Douglas A Levine, Adrian V Lee, et al. An integrated TCGA pan-cancer clinical data resource to drive high-quality survival outcome analytics. Cell, 173(2):400–416, 2018. 5
58.Ming Y Lu, Bowen Chen, Drew FK Williamson, Richard J Chen, Ivy Liang, Tong Ding, Guillaume Jaume, Igor Odintsov, Long Phi Le, Georg Gerber, et al. A visual-language foundation model for computational pathology. Nature Medicine, pages 1–12, 2024. 1
59.Ming Y Lu, Bowen Chen, Andrew Zhang, Drew FK Williamson, Richard J Chen, Tong Ding, Long Phi Le, Yung-Sung Chuang, and Faisal Mahmood. Visual language pre-trained multiple instance zero-shot transfer for histopathology images. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 19764–19775, 2023. 2
60.Ming Y Lu, Drew FK Williamson, Tiffany Y Chen, Richard J Chen, Matteo Barbieri, and Faisal Mahmood. Data-efficient and weakly supervised computational pathology on whole-slide images. Nature biomedical engineering, 5(6):555–570, 2021. 1, 2
61.Wenqi Lu, Simon Graham, Mohsin Bilal, Nasir Rajpoot, and Fayyaz Minhas. Capturing cellular topology in multi-gigapixel pathology images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pages 260–261, 2020. 2
62.Margaux Luck, Tristan Sylvain, Joseph Paul Cohen, Heloise Cardinal, Andrea Lodi, and Yoshua Bengio. Learning to rank for censored survival data. arXiv preprint arXiv:1806.01984, 2018. 2
63.Andriy Marusyk, Michalina Janiszewska, and Kornelia Polyak. Intratumor heterogeneity: the rosetta stone of therapy resistance. Cancer cell, 37(4):471–484, 2020. 7
64.Gregoire Mialon, Dexiong Chen, Alexandre d’Aspremont, and Julien Mairal. A trainable optimal transport embedding for feature aggregation and its relationship to attention. In International Conference on Learning Representations, 2021. 2, 3, 4, 5, 6, 7
65.Pooya Mobadersany, Safoora Yousefi, Mohamed Amgad, David A Gutman, Jill S Barnholtz-Sloan, Jose E Velazquez Vega, Daniel J Brat, and Lee AD Cooper. Predicting cancer outcomes from histology and genomics using convolutional networks. Proceedings of the National Academy of Sciences, 115(13):E2970–E2979, 2018. 2
66.Maxime Oquab, Timothee Darcet, Théo Moutakanni, Huy V. Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel HAZIZA, Francisco Massa, Alaaeldin El-Nouby, Mido Assran, Nicolas Ballas, Wojciech Galuba, Russell Howes, Po-Yao Huang, Shang-Wen Li, Ishan Misra, Michael Rabbat, Vasu Sharma, Gabriel Synnaeve, Hu Xu, Herve Jegou, Julien Mairal, Patrick Labatut, Armand Joulin, and Piotr Bojanowski. DINOv2: Learning robust visual features without supervision. Transactions on Machine Learning Research, 2024. 5
67.Wentao Pan, Jiangpeng Yan, Hanbo Chen, Jiawei Yang, Zhe Xu, Xiu Li, and Jianhua Yao. Human-machine interactive tissue prototype learning for label-efficient histopathology image segmentation. In International Conference on Information Processing in Medical Imaging, pages 679–691. Springer, 2023. 3
68.Shraman Pramanick, Li Jing, Sayan Nag, Jiachen Zhu, Hardik J Shah, Yann LeCun, and Rama Chellappa. VoLTA: Vision-language transformer with weakly-supervised local-feature alignment. Transactions on Machine Learning Research, 2023. 2
69.Adalberto Claudio Quiros, Nicolas Coudray, Anna Yeaton, Xinyu Yang, Bojing Liu, Hortense Le, Luis Chiriboga, Afreen Karimkhan, Navneet Narula, David A. Moore, Christopher Y. Park, Harvey Pass, Andre L. Moreira, John Le Quesne, Aristotelis Tsirigos, and Ke Yuan. Mapping the landscape of histomorphological cancer phenotypes using self-supervised learning on unlabeled, unannotated pathology slides, 2023. 1, 3, 5, 6, 7
70.Thomas Roetzer-Pejrimovsky, Anna-Christina Moser, Baran Atli, Clemens Christian Vogel, Petra A Mercea, Romana Prihoda, Ellen Gelpi, Christine Haberler, Romana Hoftberger, Johannes A Hainfellner, et al. The digital brain tumour atlas, an open histopathology resource. Scientific Data, 9(1):55, 2022. 5, 2
71.Joel Saltz, Rajarsi Gupta, Le Hou, Tahsin Kurc, Pankaj Singh, Vu Nguyen, Dimitris Samaras, Kenneth R Shroyer, Tianhao Zhao, Rebecca Batiste, et al. Spatial organization and molecular correlation of tumor-infiltrating lymphocytes using deep learning on pathology images. Cell reports, 23(1):181–193, 2018. 2
72.Geoffrey Schiebinger, Jian Shu, Marcin Tabaka, Brian Cleary, Vidya Subramanian, Aryeh Solomon, Joshua Gould, Siyan Liu, Stacie Lin, Peter Berube, et al. Optimal-transport analysis of single-cell gene expression identifies developmental trajectories in reprogramming. Cell, 176(4):928–943, 2019. 2
73.Zhuchen Shao, Hao Bian, Yang Chen, Yifeng Wang, Jian Zhang, Xiangyang Ji, et al. Transmil: Transformer based correlated multiple instance learning for whole slide image classification. Advances in Neural Information Processing Systems, 34:2136–2147, 2021. 1, 2, 5, 6, 7
74.Sivic and Zisserman. Video google: A text retrieval approach to object matching in videos. In Proceedings ninth IEEE international conference on computer vision, pages 1470–1477. IEEE, 2003. 3
75.Ole-Johan Skrede, Sepp De Raedt, Andreas Kleppe, Tarjei S Hveem, Knut Liestøl, John Maddison, Hanne A Askautrud, Manohar Pradhan, John Arne Nesheim, Fritz Albregtsen, et al. Deep learning for prediction of colorectal cancer outcome: a discovery and validation study. The Lancet, 395(10221):350–360, 2020. 2
76.Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical networks for few-shot learning. Advances in neural information processing systems, 30, 2017. 3
77.Andrew H Song, Guillaume Jaume, Drew FK Williamson, Ming Y Lu, Anurag Vaidya, Tiffany R Miller, and Faisal Mahmood. Artificial intelligence for digital and computational pathology. Nature Reviews Bioengineering, 1(12):930–949, 2023. 1
78.Wenhao Tang, Sheng Huang, Xiaoxian Zhang, Fengtao Zhou, Yi Zhang, and Bo Liu. Multiple instance learning framework with masked hard instance mining for whole slide image classification. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4078–4087, 2023. 2
79.Kevin Thandiackal, Boqi Chen, Pushpak Pati, Guillaume Jaume, Drew FK Williamson, Maria Gabrani, and Orcun Goksel. Differentiable zooming for multiple instance learning on whole-slide images. In European Conference on Computer Vision, pages 699–715. Springer, 2022. 2
80.Ilio Vitale, Efrat Shema, Sherene Loi, and Lorenzo Galluzzi. Intratumoral heterogeneity in cancer progression and response to immunotherapy. Nature Medicine, 27(2):212–224, 2021. 7
81.Quoc Dang Vu, Kashif Rajpoot, Shan E. Ahmed Raza, and Nasir Rajpoot. Handcrafted histological transformer (h2t): Unsupervised representation of whole slide images. Medical Image Analysis, 85:102743, 2023. 1, 3, 5, 6, 7
82.Tiep Huu Vu, Hojjat Seyed Mousavi, Vishal Monga, Ganesh Rao, and UK Arvind Rao. Histopathological image classification using discriminative feature-oriented dictionary learning. IEEE transactions on medical imaging, 35(3):738–751, 2015. 3
83.Xiyue Wang, Sen Yang, Jun Zhang, Minghui Wang, Jing Zhang, Wei Yang, Junzhou Huang, and Xiao Han. Transformer-based unsupervised contrastive learning for histopathological image classification. Medical image analysis, 81:102559, 2022. 1, 5, 2
84.Jinxi Xiang and Jun Zhang. Exploring low-rank property in multiple instance learning for whole slide image classification. In The Eleventh International Conference on Learning Representations, 2023. 2, 5, 6, 7
85.Yingxue Xu and Hao Chen. Multimodal optimal transport-based co-attention transformer with global structure consistency for survival prediction. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 21241–21251, 2023. 2
86.Yan Xu, Jun-Yan Zhu, Eric Chang, and Zhuowen Tu. Multiple clustered instance learning for histopathology cancer image classification, segmentation and clustering. In 2012 IEEE Conference on Computer Vision and Pattern Recognition, pages 964–971. IEEE, 2012. 3
87.Litao Yang, Deval Mehta, Sidong Liu, Dwarikanath Mahapatra, Antonio Di Ieva, and Zongyuan Ge. TPMIL: Trainable prototype enhanced multiple instance learning for whole slide image classification. In Medical Imaging with Deep Learning, 2023. 3
88.Jiawen Yao, Xinliang Zhu, Jitendra Jonnagaddala, Nicholas Hawkins, and Junzhou Huang. Whole slide images based cancer survival prediction using attention guided deep multiple instance learning networks. Medical Image Analysis, 65:101789, 2020. 2, 3, 5, 6, 7
89.Hangting Ye, Wei Fan, Xiaozhuang Song, Shun Zheng, He Zhao, Dan dan Guo, and Yi Chang. PTaRL: Prototype-based tabular representation learning via space calibration. In The Twelfth International Conference on Learning Representations, 2024. 2
90.Anna Yeaton, Rahul G Krishnan, Rebecca Mieloszyk, David Alvarez-Melis, and Grace Huynh. Hierarchical optimal transport for comparing histopathology datasets. In Medical Imaging with Deep Learning, 2022. 2
91.Jin-Gang Yu, Zihao Wu, Yu Ming, Shule Deng, Yuanqing Li, Caifeng Ou, Chunjiang He, Baiye Wang, Pusheng Zhang, and Yu Wang. Prototypical multiple instance learning for predicting lymph node metastasis of breast cancer from whole-slide pathological images. Medical Image Analysis, 85:102748, 2023. 3, 5, 6, 7
92.Shekoufeh Gorgi Zadeh and Matthias Schmid. Bias in cross-entropy-based training of deep survival networks. IEEE transactions on pattern analysis and machine intelligence, 43(9):3126–3137, 2020. 2
93.Manzil Zaheer, Satwik Kottur, Siamak Ravanbakhsh, Barnabas Poczos, Russ R Salakhutdinov, and Alexander J Smola. Deep sets. Advances in neural information processing systems, 30, 2017. 5, 6, 7
94.Hongrun Zhang, Yanda Meng, Yitian Zhao, Yihong Qiao, Xiaoyun Yang, Sarah E Coupland, and Yalin Zheng. Dtfd-mil: Double-tier feature distillation multiple instance learning for histopathology whole slide image classification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18802–18812, 2022. 2
95.Jianguo Zhang, Marcin Marszałek, Svetlana Lazebnik, and Cordelia Schmid. Local features and kernels for classification of texture and object categories: A comprehensive study. International journal of computer vision, 73:213–238, 2007. 3
96.Xinliang Zhu, Jiawen Yao, Feiyun Zhu, and Junzhou Huang. Wsisa: Making survival prediction from whole slide histopathological images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7234–7242, 2017. 2

Citation

MLA

Song, A. H., et al. “Morphological Prototyping for Unsupervised Slide Representation Learning in Computational Pathology”. arXiv, 2024, http://arxiv.org/abs/2405.11643v1.

APA

Song, A. H., Chen, R. J., Ding, T., Williamson, D. F. K., Jaume, G., & Mahmood, F. (2024). Morphological Prototyping for Unsupervised Slide Representation Learning in Computational Pathology. arXiv. http://arxiv.org/abs/2405.11643v1

Chicago

Song, A. H., R. J. Chen, T. Ding, D. F. K. Williamson, G. Jaume, and F. Mahmood. 2024. “Morphological Prototyping for Unsupervised Slide Representation Learning in Computational Pathology”. arXiv. http://arxiv.org/abs/2405.11643v1.

Harvard

Song, A.H. et al. (2024) “Morphological Prototyping for Unsupervised Slide Representation Learning in Computational Pathology”, arXiv [Preprint]. Available at: http://arxiv.org/abs/2405.11643v1.

Vancouver

1. Song AH, Chen RJ, Ding T, Williamson DFK, Jaume G, Mahmood F (2024) Morphological Prototyping for Unsupervised Slide Representation Learning in Computational Pathology. arXiv

BibTeX

@article{song2024morphological,
  title = {Morphological Prototyping for Unsupervised Slide Representation Learning in Computational Pathology},
  author = {Song, Andrew H. and Chen, Richard J. and Ding, Tong and Williamson, Drew F. K. and Jaume, Guillaume and Mahmood, Faisal},
  year = {2024},
  journal = {arXiv},
  url = {http://arxiv.org/abs/2405.11643v1},
  eprint = {2405.11643}
}

Metadata:arXiv

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF

License: IEEE

Morphological Prototyping for Unsupervised Slide Representation Learning in Computational Pathology

Abstract

Topics

Keywords

Table of Contents

Knowls

Knowl 1 — PANTHER represents a whole-slide image as Gaussian morphological prototypes

Knowl 2 — GMM fitting provides soft patch-to-prototype assignments and the slide representation

Knowl 3 — Prototype-wise nonlinear prediction uses a structured MLP

Knowl 4 — PANTHER is a nonuniform prototype aggregation related to optimal transport

Knowl 5 — Prototype assignment maps make the learned representation spatially interpretable

Knowl 6 — Experimental evaluation covers four subtyping tasks and nine survival evaluations

Knowl 7 — PANTHER achieves competitive subtyping results, with full parameter concatenation outperforming reduced variants

Knowl 8 — PANTHER improves or matches baselines across diverse disease-specific survival tasks

Knowl 9 — Ablations show that preserving all morphological components and their identities is essential

Knowl 10 — Learned prototypes correspond to clinically meaningful tissue morphologies

Knowl 11 — Fixed prototype count is a stated limitation of PANTHER

References

Citation

BibTeX

Access the Paper