Repository navigation
llms/huggingface: move default URL to router.huggingface.co/hf-inference - #1564
Open
torrresagus wants to merge 2 commits into
Open
torrresagus wants to merge 2 commits into
torrresagus wants to merge 2 commits into
Conversation
The Hugging Face LLM's default base URL now points to HF Inference at https://router.huggingface.co/hf-inference, so text-generation and embeddings requests go to `{url}/models/{model}` on the new host. I also changed the client test's base URL to match. The chat completions path already used router.huggingface.co, so it needed no change. Notice huggingface-serverless-inference-api-retired about huggingface.co's API describes the change. It was read from a file and is not signed by the provider.
The three httprr recordings still had requests to api-inference.huggingface.co, so the replay found no match once the client called router.huggingface.co/hf-inference. Only the request line, the Host header and the length prefix change; every recorded response stays byte for byte as it was recorded. Re-recording against the new host needs a Hugging Face token.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed in the API
Notice
huggingface-serverless-inference-api-retiredabouthuggingface.co's API describes this change, published 2025-07-01. It is not signed byhuggingface.co: it was read from a file the person running Tiltmeter trusts.The notice's summary:
POST https://api-inference.huggingface.co/models/{model}$POST https://router.huggingface.co/hf-inference/models/{model}POST https://api-inference.huggingface.co/v1/chat/completions$POST https://router.huggingface.co/v1/chat/completionsThe notice's notes:
What this pull request changes
The Hugging Face LLM's default base URL now points to HF Inference at https://router.huggingface.co/hf-inference, so text-generation and embeddings requests go to
{url}/models/{model}on the new host. I also changed the client test's base URL to match. The chat completions path already used router.huggingface.co, so it needed no change.llms/huggingface/huggingfacellm_option.gollms/huggingface/internal/huggingfaceclient/huggingfaceclient_test.goLeft for a human
go test -httprecord=. ./llms/huggingface/...). I didn't hand-edit them because that would make up responses from the new host.gpt2and theHuggingFaceH4/zephyr-7b-betaused in tests may not be served at router.huggingface.co/hf-inference anymore. Large chat models go through router.huggingface.co/v1/chat/completions. Choosing a new default model or changing the default routing is a product decision, so I left both as they are.This pull request was generated automatically by Tiltmeter from the unsigned notice
huggingface-serverless-inference-api-retiredabout huggingface.co's API. An AI agent (Claude Code with claude-opus-5-5) wrote these edits; review them like any other change.Test recordings
A second commit points the three httprr recordings (
TestClient_RunInference,TestClient_RunInferenceText2Text,TestHuggingFaceLLMStandardInference) at the new URL. Only each request line,Hostheader and length prefix change; the recorded responses are untouched. Re-recording them against router.huggingface.co needs a Hugging Face token, so I left that to you.