Autoencoder-based sequence embedding

[grst]

Description of feature

IMO autoencoder-based sequence embedding has a huge potential for finding similar immune receptors, potentially improving both the speed and the accuracy compared to alignment-based metrics. In particular, finding similar sequences is important in two scirpy functions:

• defining clonotypes
• querying immune receptor databases.

For the database query, an online-update algorithm similar to scArches for gene expression would be nice: The autoencoder could be trained on the database (which might have millions of unique receptors) once. A new dataset (which might only have 10k-100k unique receptors), could be projected into the same latent space as the database, significantly improving query time.

An extension to this idea is to embed gene expression and TCR/BCR data into the same latent space.

Existing tools

• Trex by @ncborcherding. Based on keras.
• mvTCR by @b-schubert's lab. Combines receptor/Gex data. Based on pytorch.
• TESSA. Combines receptor/Gex data. Not even sure it's an autoencoder, need yet to check in detail, but it seems to use some clever sequence embeddings.
• There are likely more...

---

@drEast mentioned he is working on something like that a few months ago. Are you willing to share a few details and if you would be interested in integrating it with scirpy? @adamgayoso, any chance there's AirrVI soon? :stuck_out_tongue_winking_eye:

[FFinotello]

Thanks, @grst for putting this together! It might be worth embedding also HLA type information (see also this recent publication).

[ncborcherding]

Agreed deepTCR from @sidhomj would be a good addition to the list.

[drEast]

Happy to help integrating mvTCR into scirpy / sc-verse. Would you prefer having external packages with interfaces to Scirpy, or integrating them directly into the main package? The latter will potentially cause quite some dependencies-issues between the different tools.

As you mentioned, we are also currently working on a TCR embedding that is guided by antigen specificity (in contrast to most AE-based methods) and, therefore, potentially better clusters TCRs by their target epitope. However, development is still ongoing.

[grst]

I had a chat with @drEast yesterday, summarizing the main points below:

• mvTCR integrates exactly one pair of alpha/beta chains per cell with gene expression data
• it does not take into account V/D genes or HLA types
• It works in principle to integrate BCR data as well, but it is not validated. It does not make sense to have BCR/TCR in the same model.
• After submitting their preprint, they will work with me on making it compatible with scirpy as external package.

For the future the model could be extended

• to become more flexible, to optionally account for secondary chains and/or V/D genes
• return the TCR-space independently of the gene expression space

[b-schubert]

Re future directions:

• theoretically, in mvTCR you also have independent transcriptomic and TCR spaces, though we have not fully explored their structures yet.
• I am teaching a single-cell analysis course this winter semester with student projects where we are going to explore VDJ encodings and pre-trained TCR models in mvTCR, we could also explore the options of how to integrate HLA types (more than just as confounding)

[grst]

theoretically, in mvTCR you also have independent transcriptomic and TCR spaces, though we have not fully explored their structures yet.

As I understood @drEast yesterday, as it is currently implemented, it cannot be ran independently of the transcriptomic module and it's also not possible to directly access the TCR space. But maybe that's just a minor implementation detail.

[FFinotello]

Hello everyone, and thanks for this great exchange! :)

For the HLA types, it would be great to keep track somehow of their sequence similarity. We could also consider their level of expression, at least broadly assigned to HLA-A, HLA-B, and HLA-C genes, whereas allele-specific expression would be hard to derive from 10x data. But @b-schubert you are definitely the expert here :)

[michael-swift]

hey folks, I was thinking along similar lines and used ablang to investigate whether these LLMs are any good for looking at BCRs. see the post here: https://michael-swift.github.io/posts/ I'm happy to help pull something useful into scirpy, lmk

[grst]

Hi @michael-swift,

this is super cool! For interoperability with scirpy, the preferred way would now be to make a separate package that operates on an AnnData object with the scirpy data structure. To get started, I recommend checking out our cookiecutter template.

I am happy to promote such a package via the scirpy documentation and the scverse ecosystem page.

Also happy to help with any questions regarding how to make such a package fully interoperable with scirpy.

[michael-swift]

thanks for the info! I'll look into making a package