opea-project
[SearchQnA] [ChatQnA] TEI service config for 1-card Gaudi scenario
When using single-card Gaudi, We would like to make changes to docker/gaudi/docker/docker_compose.yaml, as shown in the picture below, to move TEI embedding microservice to CPU. This is due to the issue reported in #370
Need that information in the readme, or make corresponding changes to detect the number of HPU cards and pick the right version of docker_compose.yaml.
to move TEI embedding microservice to CPU
Why? Is TEI-embedding Gaudi utilization too low for it to make sense, or is there some other reason?
to move TEI embedding microservice to CPU
Why? Is TEI-embedding Gaudi utilization too low for it to make sense, or is there some other reason?
Please refer to the problem reported in #370
You can change the docker compose file according your system resource. So it's not an issue.
