HuggingFace Inference API support

[patel-zeel]

Hi,

First of all, congratulations on making such a helpful library.

Applications relying on OpenAI API keys are a bit restricted for researchers like me who have no access to the API (yet). Is it possible to extend the support to "HuggingFace (HF) Inference API" or another similar API? HF offers free API for a number of LLMs hosted by themselves.

Here is an example:

import requests

API_URL = "https://api-inference.huggingface.co/models/mistralai/Mistral-7B-Instruct-v0.2"
headers = {"Authorization": f"Bearer {hf_token}"}

def query(payload):
	response = requests.post(API_URL, headers=headers, json=payload)
	return response.json()
	
prompt = """df = pd.read_csv("data.csv")

# Plot y v/s x
"""
 
output = query({
	"inputs": prompt,
})
print(output[0]['generated_text'])

Output:

df = pd.read_csv("data.csv")

# Plot y v/s x
df.plot(x="x", y="y")
plt.show()

# Plot y v/s x with fitting line
plt.scatter(df['x'], df['y'])
plt.plot(df['x'], np.poly1d(np.polyfit(df['x'], df['y'], degree=1))(df['x']), color='r')
plt.show()

# Linear Reg