colpali
colpali copied to clipboard
illuin-tech
The code used to train and run inference with the ColVision models, e.g. ColPali, ColQwen2, and ColSmol.
## Description ### Added - Restore, refactor, and improve the `interpretability` module for generating similarity maps ## Tests The following figures were obtained using the code from https://github.com/tonywu71/colpali-cookbooks/pull/5. ### ColPali...
Hey guys thanks for the fantastic project. I just need some clarification on how I initialize the model. When I initialize the model using the example provided on HF, and...
I am using the vidore/colqwen2-v0.1 model to embed documents and queries. In the code, I noticed that you removed pixel_values during the query processing, but you didn't remove the described...
When executing the following command: `USE_LOCAL_DATASET=0 python scripts/train/train_colbert.py scripts/configs/pali/train_colpali_docmatix_model.yaml` I encounter the following error message: `Processor should be provided for vision collator` My pretrained_model_name_or_path is: `/tmp2/vik/cache/models--vidore--colpaligemma-3b-mix-448-base/snapshots/6ff0d944ea09c3ead97d2bc57427e3d4f01d192f` What might I be...
## Description Implement and train ColPali2. The most noticeable feature of ColPali2 is to being able to output both single-vector (dense) and multi-vector (ColBERT-like) embeddings. Follow-up of https://github.com/illuin-tech/colpali/pull/22.
Currently, many things are hacked around, which makes `colpali-engine` frustrating to build on. The abstractions of the following need to be improved: - tokenization via AutoProcessor. Queries cannot be processed...
I still don't know if the Colpali model is only contextual to each or all documents. If only contextual to each document, then we could integrate late chunking to maximize...
Can you please share the PDFs sources which used to build the vidore benchmark Thanks
Metadata
Owner
Metadata
The code used to train and run inference with the ColVision models, e.g. ColPali, ColQwen2, and ColSmol.