mahmoodlab
Generalization to another anatomical sites
Really nice work, congrats!
(1) What are your feelings about the generalization capability of MADELEINE (w/o fine-tuning) to another anatomical sites different to breast and kidney? I mean, using MADELEINE to get the slide representation of lung WSIs from TCGA-NSCLC. A quick experimentation on linear probing comparing CONCH + BGAP and MADELINE features in fully-supervised setting (w/o few-shot) problem of cancer subtyping (LUAD vs LUSC) is (at least) showing slight decrease in performance: 89.92% and 88.49% in terms of balanced accuracy.
(2) What is the motivation of not experimenting MADELEINE with kidney samples for cancer subtyping in TCGA-RCC? The same experimentation for this task I am getting 91.04% BACC for CONCH+BGAP and 91.50% BACC for MADELEINE. Also making some few-shot learning experiments (k=4, five seeds), I am getting a decrease of 1.34% of accuracy.
(3) Finally, why do you think that the superior performance of MADELEINE against other MIL models is more prominent in low-shot setting (k=1 or k=5) than in high-shot settings (k=10 or k=25)?
Hi @PabloMeseguerEsbri, thanks for your interest in our work.
-
MADELEINE is trained on breast cancer/kidney rejection, so it may not generalize well to sites like lung (as it does not have lung morphology aligned with lung specific stains). I expect CONCH mean pooling to perform similarly to MADELEINE on sites outside its training set.
-
MADELEINE-Kidney is trained on non-cancer rejection slides and lacks understanding of cancerous kidney tissue. However, with larger multi-stain kidney cancer cohorts, you can use our codebase to train your own MADELEINE-Kidney model.
-
MIL with a strong patch encoder like CONCH is a solid baseline with ample data, but in low-data scenarios, MIL can overfit, possibly leading to worse performance. MADELEINE and similar slide encoders may offer advantages here, as they are task-agnostic, pre-trainable on large datasets, and less prone to overfitting in low-data settings.
I hope these answer your questions, but feel free to post any additional queries.
