Fanli Lin
Fanli Lin
it works by adding `sharding=RayShardingMode.BATCH` to the prediction `RayDMatrix`. Close this issue.
@vblagoje any feedback on this PR?
@vblagoje fyi, I removed the parameters that will modify the model config files and only include those that will affect the generation behavior, see this blog post here: https://huggingface.co/blog/how-to-generate .
> @faaany , thanks for your patience. If possible, I'd like to remove this filter altogether. It was put in there back in the day when a single unrecognized parameter...
@vblagoje I have updated my PR and removed the key argument filter. But I need to pop `documents` and `query` from the list, otherwise the unit tests `` in `test_prompt_node.py`...
@vblagoje In my opinion, it is the user's responsibility to pass the correct parameters to the pipeline. If we want to assist users in passing the correct parameters, we could...
@vblagoje And to your proposal that we could add a more generic method to remove all prompt template parameters, I am not sure about this, because usually the documents and...
> @faaany I looked at this issue in detail and concluded the following. We currently pass all template input kwargs to the invocation layer invoke method. And we shouldn't do...
@vblagoje I have updated the code. Could you take a look?