tranquanghust

Is the instruction model in your project a bi-encoder? Can the cross-encoder be fine-tuned in the direction of 'Instruction-Finetuned Text' like that?

How to mine hard neg for reranker?

1

I just saw the script that mine the hard negative for the bi encoder

something is wrong with hard neg mine

2

I tried hard neg mining, but when running on 2 different gpu's (namely t4 and a100), t4 only took a few seconds while a100 took 20 minutes, and their hard...

ValueError: Attempting to unscale FP16 gradients.

4

Here is the Google Colab link I used for fine-tuning : [https://colab.research.google.com/drive/1kiALBR1UarPobiftZmiHfwFyk7hTCDnV?usp=sharing](url) When I fine-tune the LLM-embed for tool retrieval using the command on Google Colab:  An error occurred:...

how to adjust hyperparameter for finetune llm embed

2

llm embed has the following training script. I don't know how to adjust hyperparameters like train_batch_size, learning rate, warmup_ratio, ... torchrun --nproc_per_node=8 run_dense.py \ --output_dir data/outputs/tool \ --train_data llm-embedder:tool/toolbench/train.json \...

bge for multi-label text classification

2

Is there any BGE model that can be used for multi-label text classification (predefined labels) by adding some dense layers for classification (similar to BERT base uncased) instead of Sentence...

finetune instruction for bge rerank

5

Can BGE rerank be finetuned in an instruction style, for example: "tool": { "query": "Transform this user request for fetching helpful tool descriptions: ", "key": "Transform this tool description for...

Regarding the issue of fine-tuning on a specific domain

8

Dear author, has the file examples/finetune.ipynb included negative entity sampling yet? If not, how can we adjust it to incorporate negative entity sampling?