promptfoo
deprecate vertex models not supported by provider
Is your feature request related to a problem? Please describe. Docs list Vertex models which have been deprecated. See deprecation notice of Vertex PaLM models as of April 9, 2025.
Describe the solution you'd like Remove the following models from the site/docs/providers/vertex.md page
- `vertex:chat-bison`
- `vertex:chat-bison@001`
- `vertex:chat-bison@002`
- `vertex:chat-bison-32k`
- `vertex:chat-bison-32k@001`
- `vertex:chat-bison-32k@002`
- `vertex:codechat-bison`
- `vertex:codechat-bison@001`
- `vertex:codechat-bison@002`
- `vertex:codechat-bison-32k`
- `vertex:codechat-bison-32k@001`
- `vertex:codechat-bison-32k@002`
Describe alternatives you've considered Ignoring that chunk of documentation?
Additional context Given that the discontinuation date is April 9, 2025, can wait to submit a PR for this change. PTAL and let me know, thank you!
@typpo I was thinking of possibly moving them below the current Gemini section with a note "These models are scheduled for deprecation (documentation link ref)" for now and then post April another PR to remove completely? (We do still see some customers using the PaLM models, but not sure if those overlap with people using promptfoo) - open to ideas here!
That sounds reasonable to me :)
On Thu, Feb 20, 2025 at 12:05 PM G. Hussain Chinoy @.***> wrote:
@typpo https://github.com/typpo I was thinking of possibly moving them below the current Gemini section with a note "These models are scheduled for deprecation (documentation link ref)" for now and then post April another PR to remove completely? (We do still see some customers using the PaLM models, but not sure if those overlap with people using promptfoo) - open to ideas here!
— Reply to this email directly, view it on GitHub https://github.com/promptfoo/promptfoo/issues/3169#issuecomment-2672557084, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACLYJWCCBIEXVI6FRWDPBD2QYYPPAVCNFSM6AAAAABXRLVWMOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMNZSGU2TOMBYGQ . You are receiving this because you were mentioned.Message ID: @.***> [image: ghchinoy]ghchinoy left a comment (promptfoo/promptfoo#3169) https://github.com/promptfoo/promptfoo/issues/3169#issuecomment-2672557084
@typpo https://github.com/typpo I was thinking of possibly moving them below the current Gemini section with a note "These models are scheduled for deprecation (documentation link ref)" for now and then post April another PR to remove completely? (We do still see some customers using the PaLM models, but not sure if those overlap with people using promptfoo) - open to ideas here!
— Reply to this email directly, view it on GitHub https://github.com/promptfoo/promptfoo/issues/3169#issuecomment-2672557084, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACLYJWCCBIEXVI6FRWDPBD2QYYPPAVCNFSM6AAAAABXRLVWMOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMNZSGU2TOMBYGQ . You are receiving this because you were mentioned.Message ID: @.***>
@ghchinoy That's a great idea. I started to update the vertex docs in https://github.com/promptfoo/promptfoo/pull/3107 but fell into a rabbit hole when I realized we needed to improve compatibility for claude in vertex. You are free to pull from that branch / take it over / or I can promise to try to land it within the next few days.
@mldangelo Ah, yes, including all the available Model Garden models from Vertex. Makes sense - and this change is minor relative to #3107 - I'm happy to wait at your convenience.