KBLaM
KBLaM copied to clipboard
microsoft
Official Implementation of "KBLaM: Knowledge Base augmented Language Model"
Training as proposed in the README or executing the 3rd cell of the eval_acc.ipynb gives the following error: File ~/Thomas/KBLaM/src/kblam/models/llama3_model.py:43 38 from transformers.modeling_outputs import ( 39 BaseModelOutputWithPast, 40 CausalLMOutputWithPast, 41...
Bumps the dependency-packages group with 2 updates in the / directory: [datasets](https://github.com/huggingface/datasets) and [transformers](https://github.com/huggingface/transformers). Updates `datasets` from 3.2.0 to 3.3.2 Release notes Sourced from datasets's releases. 3.3.2 Bug fixes Attempt...
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot] avatar"){kind=avatar}
It seems I'm missing some modules when I run the experiements: ``` ~/kblam/experiments$ python train.py --verbose --dataset synthetic_data --N 120000 --B 20 --total_steps 601 --encoder_spec OAI --use_oai_embd --key_embd_src key --use_data_aug...
how does the model knows, if need to apply adaptor or not
The assumption each line is it's own DataPoint is untrue, and breaks when trying to use the example datasets.
This classes are deprecated at transformers==4.46: ```python LlamaDynamicNTKScalingRotaryEmbedding LlamaLinearScalingRotaryEmbedding "`LlamaLinearScalingRotaryEmbedding` is deprecated an will be removed in v4.46. Please use " ``` However, KBLaM depends on transformers==4.48.0 and use that...
Hi @ti250 🤗 Niels here from the open-source team at Hugging Face. I discovered your work on Arxiv and was wondering whether you would like to submit it to [hf.co/papers](https://hf.co/papers)...
Hello, Thank you for sharing KBLaM's source code. I'd like to know more about the reproducibility of this work on different knowledge bases. According to what I've read, these are...
Trying Phi3 instead of Llama3 for the training with python train.py --dataset_dir ../datasets --train_dataset enron --N 120000 --B 20 --total_steps 601 --encoder_spec OAI --use_oai_embd --key_embd_src key --use_data_aug --hf_token ... --llm_type...
It seems that the configuration information in 'meta-llama/Llama-3.2-1B-Instruct/resolve/main/config.json' has changed since the code was used the last time. Running the training on the enron dataset gives: ` File "/home/fokus/Thomas/KBLaM/src/kblam/models/llama3_model.py", line...