Aastha Varma comments

Results 5 comments of


                                            Aastha Varma

Langchain isn't verbose any more [run_manager gets dropped]

+1 on the issue with verbose=True not helping with printing the Query generated. Although line 175 in [code](https://github.com/langchain-ai/langchain/blob/master/libs/langchain/langchain/retrievers/self_query/base.py#L175) clearly shows it should output the query generated, I am not entirely...

Inference in PyTorch, what do the wrappers mean? What’s best? | fastblog

It was an interesting read !

Does deepspeed inference support PTQ?

Interested in this discussion - Any updates on this ?

[QUESTION] How to figure out correct `injection_policy` for Flan-T5

Thanks @brevity2021 for the [doc](https://github.com/microsoft/DeepSpeed/blob/4ae3a3da0dfd19d7ab7a76e7c742ac12f44fc1c0/docs/_tutorials/automatic-tensor-parallelism.md). I want to use Flan-UL2 model, but didn't find that in neither supported / unsupported models. Can anyone help figure out the correct injection_policy for...

ValueError: FlatParameter requires uniform dtype but got torch.float16 and torch.float32

I tried launching the script with `FSDP_CPU_RAM_EFFICIENT_LOADING=1` but didn't work . Having same [issue](https://github.com/huggingface/accelerate/issues/1620#issuecomment-2058021031). This is the [blog](https://www.philschmid.de/fsdp-qlora-llama3) I am following. My command: `FSDP_CPU_RAM_EFFICIENT_LOADING=1 torchrun --nproc_per_node=4 run_fsdp_qlora.py --config config.yaml` These...