donguyen32
donguyen32
@abetlen Sorry but do you have any plans to implement this?
@yutyan0119 Arcoding from the original repo, I see that the format of the rerank task is `[BOS]query[EOS][SEP]doc[EOS]` https://github.com/ggerganov/llama.cpp/blob/9f409893519b4a6def46ef80cd6f5d05ac0fb157/examples/server/utils.hpp#L185-L196 your inputs are` [f"{query}{doc}" for doc in documents]` Please check it
@KanishkNavale @yazon @handshape Please view PR https://github.com/abetlen/llama-cpp-python/pull/1820 I think it works well