guidance
guidance copied to clipboard
microsoft
TimeoutError when running the provided example with VPN
Hello maintainers of the Guidance repository,
Firstly, I want to express my appreciation for the work you've done with Guidance. It's an incredibly useful library, and I'm grateful for the effort you've put into it.
Unfortunately, I encountered an issue when trying to run the provided example from the documentation. Here are the details:
Steps to Reproduce: Ran the example provided by the authors:
import guidance
llm = guidance.llms.OpenAI('text-davinci-003', api_key='MY_API_KEY')
program = guidance("""My favorite flavor is{{gen 'flavor' max_tokens=10 stop="."}}""", llm=llm)
program()
Actual Result:
I received a TimeoutError:
Traceback (most recent call last):
File "/Users/xxx/opt/anaconda3/envs/python39/lib/python3.9/site-packages/aiohttp/connector.py", line 980, in _wrap_create_connection
return await self._loop.create_connection(*args, **kwargs) # type: ignore[return-value] # noqa
File "/Users/xxx/opt/anaconda3/envs/python39/lib/python3.9/asyncio/base_events.py", line 1065, in create_connection
raise exceptions[0]
File "/Users/xxx/opt/anaconda3/envs/python39/lib/python3.9/asyncio/base_events.py", line 1050, in create_connection
sock = await self._connect_sock(
File "/Users/xxx/opt/anaconda3/envs/python39/lib/python3.9/asyncio/base_events.py", line 961, in _connect_sock
await self.sock_connect(sock, address)
File "/Users/xxx/opt/anaconda3/envs/python39/lib/python3.9/asyncio/selector_events.py", line 500, in sock_connect
return await fut
File "/Users/xxx/opt/anaconda3/envs/python39/lib/python3.9/asyncio/futures.py", line 284, in __await__
yield self # This tells Task to wait for completion.
File "/Users/xxx/opt/anaconda3/envs/python39/lib/python3.9/asyncio/tasks.py", line 328, in __wakeup
future.result()
File "/Users/xxx/opt/anaconda3/envs/python39/lib/python3.9/asyncio/futures.py", line 201, in result
raise self._exception
File "/Users/xxx/opt/anaconda3/envs/python39/lib/python3.9/asyncio/selector_events.py", line 535, in _sock_connect_cb
raise OSError(err, f'Connect call failed {address}')
TimeoutError: [Errno 60] Connect call failed ('199.59.148.209', 443)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/xxx/opt/anaconda3/envs/python39/lib/python3.9/site-packages/openai/api_requestor.py", line 668, in arequest_raw
result = await session.request(**request_kwargs)
File "/Users/xxx/opt/anaconda3/envs/python39/lib/python3.9/site-packages/aiohttp/client.py", line 536, in _request
conn = await self._connector.connect(
File "/Users/xxx/opt/anaconda3/envs/python39/lib/python3.9/site-packages/aiohttp/connector.py", line 540, in connect
proto = await self._create_connection(req, traces, timeout)
File "/Users/xxx/opt/anaconda3/envs/python39/lib/python3.9/site-packages/aiohttp/connector.py", line 901, in _create_connection
_, proto = await self._create_direct_connection(req, traces, timeout)
File "/Users/xxx/opt/anaconda3/envs/python39/lib/python3.9/site-packages/aiohttp/connector.py", line 1206, in _create_direct_connection
raise last_exc
File "/Users/xxx/opt/anaconda3/envs/python39/lib/python3.9/site-packages/aiohttp/connector.py", line 1175, in _create_direct_connection
transp, proto = await self._wrap_create_connection(
File "/Users/xxx/opt/anaconda3/envs/python39/lib/python3.9/site-packages/aiohttp/connector.py", line 988, in _wrap_create_connection
raise client_error(req.connection_key, exc) from exc
aiohttp.client_exceptions.ClientConnectorError: Cannot connect to host api.openai.com:443 ssl:default [Connect call failed ('199.59.148.209', 443)]
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/xxx/opt/anaconda3/envs/python39/lib/python3.9/site-packages/guidance/_program_executor.py", line 109, in run
await self.visit(self.parse_tree, VariableStack([self.program._variables], self))
File "/Users/xxx/opt/anaconda3/envs/python39/lib/python3.9/site-packages/guidance/_program_executor.py", line 559, in visit
visited_children.append(await self.visit(child, variable_stack, inner_next_node, inner_next_next_node, inner_prev_node, node, parent_node))
File "/Users/xxx/opt/anaconda3/envs/python39/lib/python3.9/site-packages/guidance/_program_executor.py", line 266, in visit
visited_children = [await self.visit(child, variable_stack, next_node, next_next_node, prev_node, node, parent_node) for child in node]
File "/Users/xxx/opt/anaconda3/envs/python39/lib/python3.9/site-packages/guidance/_program_executor.py", line 266, in <listcomp>
visited_children = [await self.visit(child, variable_stack, next_node, next_next_node, prev_node, node, parent_node) for child in node]
File "/Users/xxx/opt/anaconda3/envs/python39/lib/python3.9/site-packages/guidance/_program_executor.py", line 379, in visit
command_output = await command_function(*positional_args, **named_args)
File "/Users/xxx/opt/anaconda3/envs/python39/lib/python3.9/site-packages/guidance/library/_gen.py", line 140, in gen
gen_obj = await parser.llm_session(
File "/Users/xxx/opt/anaconda3/envs/python39/lib/python3.9/site-packages/guidance/llms/_openai.py", line 665, in __call__
out = await self.llm.caller(**call_args)
File "/Users/xxx/opt/anaconda3/envs/python39/lib/python3.9/site-packages/guidance/llms/_openai.py", line 373, in _library_call
out = await openai.Completion.acreate(**kwargs)
File "/Users/xxx/opt/anaconda3/envs/python39/lib/python3.9/site-packages/openai/api_resources/completion.py", line 45, in acreate
return await super().acreate(*args, **kwargs)
File "/Users/xxx/opt/anaconda3/envs/python39/lib/python3.9/site-packages/openai/api_resources/abstract/engine_api_resource.py", line 217, in acreate
response, _, api_key = await requestor.arequest(
File "/Users/xxx/opt/anaconda3/envs/python39/lib/python3.9/site-packages/openai/api_requestor.py", line 372, in arequest
result = await self.arequest_raw(
File "/Users/xxx/opt/anaconda3/envs/python39/lib/python3.9/site-packages/openai/api_requestor.py", line 685, in arequest_raw
raise error.APIConnectionError("Error communicating with OpenAI") from e
openai.error.APIConnectionError: Error communicating with OpenAI
Error in program: Error communicating with OpenAI
Additional Information:
- Every time I run the program, the address in the error message changes.
- Due to certain restrictions, I must use a VPN to access Openai services. I suspected this might be the cause, so I tried to ping the address from the error message:
PING 199.59.148.209 (199.59.148.209): 56 data bytes
Request timeout for icmp_seq 0
- When I turn on my VPN, I can successfully call the Openai API.
import openai
openai.api_key = 'My_API_KEY'
naive_model = 'gpt-3.5-turbo'
completion = openai.ChatCompletion.create(
model=naive_model,
messages=[{'role': 'user', 'content': 'hello!'}],
)
response = completion.choices[0].message["content"]
print(response)
Result:
Hello! How can I assist you today?
Environment:
Operating System: MacOS
Python version: 3.9.13
Guidance version: 0.0.64
OpenAI version: 0.27.8
packages in environment at /Users/xxx/opt/anaconda3/envs/python39:
aiohttp 3.8.4 py39ha30fb19_0 conda-forge
aiosignal 1.3.1 pyhd8ed1ab_0 conda-forge
anyio 3.5.0 py39hecd8cb5_0
appnope 0.1.2 py39hecd8cb5_1001
argon2-cffi 20.1.0 py39h9ed2024_1
asttokens 2.0.5 pyhd3eb1b0_0
async-timeout 4.0.2 pyhd8ed1ab_0 conda-forge
attrs 22.1.0 py39hecd8cb5_0
backcall 0.2.0 pyhd3eb1b0_0
beautifulsoup4 4.11.1 py39hecd8cb5_0
biopython 1.81 pypi_0 pypi
blas 2.116 openblas conda-forge
blas-devel 3.9.0 16_osx64_openblas conda-forge
bleach 4.1.0 pyhd3eb1b0_0
bottleneck 1.3.5 py39h67323c0_0
brotli 1.0.9 hca72f7f_7
brotli-bin 1.0.9 hca72f7f_7
brotlipy 0.7.0 py39h9ed2024_1003
bzip2 1.0.8 h1de35cc_0
ca-certificates 2023.5.7 h8857fd0_0 conda-forge
cachetools 5.3.1 pypi_0 pypi
cairo 1.16.0 h0ab9d94_1001 conda-forge
certifi 2023.5.7 pyhd8ed1ab_0 conda-forge
cffi 1.15.1 py39hc55c11b_0
charset-normalizer 2.0.4 pyhd3eb1b0_0
click 8.1.3 pypi_0 pypi
colorama 0.4.6 pyhd8ed1ab_0 conda-forge
comm 0.1.2 py39hecd8cb5_0
cryptography 37.0.1 py39hf6deb26_0
cycler 0.11.0 pyhd3eb1b0_0
dataclasses-json 0.5.7 pyhd8ed1ab_0 conda-forge
debugpy 1.5.1 py39he9d5cce_0
decorator 5.1.1 pyhd3eb1b0_0
defusedxml 0.7.1 pyhd3eb1b0_0
diskcache 5.6.1 pypi_0 pypi
dnspython 2.3.0 pypi_0 pypi
ecos 2.0.12 pypi_0 pypi
eigen 3.4.0 h940c156_0 conda-forge
entrypoints 0.4 py39hecd8cb5_0
executing 0.8.3 pyhd3eb1b0_0
ffmpeg 4.3 h0a44026_0 pytorch
fftw 3.3.9 h9ed2024_1
filelock 3.12.2 pypi_0 pypi
fontconfig 2.13.94 h10f422b_0 conda-forge
fonttools 4.25.0 pyhd3eb1b0_0
freetype 2.11.0 hd8bbffd_0
frozenlist 1.3.3 py39ha30fb19_0 conda-forge
fsspec 2023.6.0 pypi_0 pypi
future 0.18.3 pyhd8ed1ab_0 conda-forge
gdk-pixbuf 2.42.8 h90ff1d3_1
gettext 0.21.0 h7535e17_0
giflib 5.2.1 haf1e3a3_0
glib 2.69.1 h8346a28_1
gmp 6.2.1 he9d5cce_3
gnutls 3.6.15 hed9c0bf_0
gptcache 0.1.38 pypi_0 pypi
graphite2 1.3.14 he9d5cce_1
greenlet 2.0.2 py39h7a8716b_0 conda-forge
gst-plugins-base 1.14.0 h4180768_2
gstreamer 1.14.0 h0fc69c2_2
guidance 0.0.64 pypi_0 pypi
h5py 3.7.0 py39h4a1dd59_0
harfbuzz 4.3.0 h3f129a2_0
hdf5 1.10.6 h10fe05b_1
histolab 0.6.0 pypi_0 pypi
huggingface-hub 0.15.1 pypi_0 pypi
icu 58.2 h0a44026_3
idna 3.3 pyhd3eb1b0_0
imageio 2.25.1 pypi_0 pypi
importlib-metadata 6.0.0 pypi_0 pypi
intel-openmp 2021.4.0 hecd8cb5_3538
ipykernel 6.19.2 py39h01d92e1_0
ipython 8.10.0 py39hecd8cb5_0
ipython_genutils 0.2.0 pyhd3eb1b0_1
jedi 0.18.1 py39hecd8cb5_1
jinja2 3.0.3 pyhd3eb1b0_0
joblib 1.3.1 pypi_0 pypi
jpeg 9e hca72f7f_0
jsonschema 4.17.3 py39hecd8cb5_0
jupyter_client 7.4.9 py39hecd8cb5_0
jupyter_core 5.2.0 py39hecd8cb5_0
jupyter_server 1.23.4 py39hecd8cb5_0
jupyterlab_pygments 0.1.2 py_0
kiwisolver 1.4.2 py39he9d5cce_0
krb5 1.19.3 hb49756b_0 conda-forge
lame 3.100 h1de35cc_0
langchain 0.0.190 pyhd8ed1ab_0 conda-forge
lcms2 2.12 hf1fd2bf_0
lerc 3.0 he9d5cce_0
libblas 3.9.0 16_osx64_openblas conda-forge
libbrotlicommon 1.0.9 hca72f7f_7
libbrotlidec 1.0.9 hca72f7f_7
libbrotlienc 1.0.9 hca72f7f_7
libcblas 3.9.0 16_osx64_openblas conda-forge
libclang 12.0.0 default_hbc2896b_2
libcxx 14.0.6 h9765a3e_0
libdeflate 1.8 h9ed2024_5
libedit 3.1.20191231 h0678c8f_2 conda-forge
libffi 3.3 hb1e8313_2
libgfortran 5.0.0 11_2_0_h246ff09_26
libgfortran5 11.2.0 h246ff09_26
libiconv 1.16 hca72f7f_2
libidn2 2.3.2 h9ed2024_0
liblapack 3.9.0 16_osx64_openblas conda-forge
liblapacke 3.9.0 16_osx64_openblas conda-forge
libllvm12 12.0.1 hd011deb_2 conda-forge
libopenblas 0.3.21 openmp_h947e540_2 conda-forge
libpng 1.6.37 ha441bb4_0
libpq 12.9 h1c9f633_3
libprotobuf 3.20.1 h8346a28_0
libsodium 1.0.18 h1de35cc_0
libtasn1 4.16.0 h9ed2024_0
libtiff 4.4.0 h2ef1027_0
libunistring 0.9.10 h9ed2024_0
libuv 1.44.2 hac89ed1_0 conda-forge
libwebp 1.2.2 h56c3ce4_0
libwebp-base 1.2.2 hca72f7f_0
libxml2 2.9.14 hbf8cd5e_0
libxslt 1.1.35 h5b33f42_0
llvm-openmp 14.0.6 h0dcd299_0
loguru 0.7.0 pypi_0 pypi
lxml 4.9.1 py39h65b224f_0
lz4-c 1.9.3 h23ab428_1
markupsafe 2.1.1 py39hca72f7f_0
marshmallow 3.19.0 pyhd8ed1ab_0 conda-forge
marshmallow-enum 1.5.1 pyh9f0ad1d_3 conda-forge
matplotlib 3.5.2 py39hecd8cb5_0
matplotlib-base 3.5.2 py39hfb0c5b7_0
matplotlib-inline 0.1.6 py39hecd8cb5_0
mistune 0.8.4 py39h9ed2024_1000
mkl 2021.4.0 hecd8cb5_637
mkl-service 2.4.0 py39h9ed2024_0
msal 1.23.0 pypi_0 pypi
multidict 6.0.4 py39ha30fb19_0 conda-forge
munkres 1.1.4 py_0
mypy_extensions 1.0.0 pyha770c72_0 conda-forge
nbclassic 0.5.2 py39hecd8cb5_0
nbclient 0.5.13 py39hecd8cb5_0
nbconvert 6.5.4 py39hecd8cb5_0
nbformat 5.7.0 py39hecd8cb5_0
ncurses 6.3 hca72f7f_3
nest-asyncio 1.5.6 py39hecd8cb5_0
nettle 3.7.3 h230ac6f_1
networkx 3.0 pypi_0 pypi
ninja 1.11.1 hb8565cd_0 conda-forge
nltk 3.8.1 pypi_0 pypi
notebook 6.5.2 py39hecd8cb5_0
notebook-shim 0.2.2 py39hecd8cb5_0
nspr 4.35 hea0b92c_0 conda-forge
nss 3.74 h47edf6a_0
numexpr 2.8.4 py39h75c1baf_0 conda-forge
numpy 1.24.2 py39h6ee2318_0 conda-forge
openai 0.27.8 pypi_0 pypi
openapi-schema-pydantic 1.2.4 pyhd8ed1ab_0 conda-forge
openblas 0.3.21 openmp_h2bb6d6c_2 conda-forge
opencv 4.6.0 py39h0e6eb04_2
openh264 2.1.1 h8346a28_0
openjpeg 2.3.1 h6cbf5cd_3 conda-forge
openslide 3.4.1 h17b48f0_1 conda-forge
openslide-python 1.2.0 pypi_0 pypi
openssl 1.1.1u h8a1eda9_0 conda-forge
osqp 0.6.3 pypi_0 pypi
packaging 21.3 pyhd3eb1b0_0
pandas 1.4.3 py39he9d5cce_0
pandocfilters 1.5.0 pyhd3eb1b0_0
parso 0.8.3 pyhd3eb1b0_0
pcre 8.45 he49afe7_0 conda-forge
pexpect 4.8.0 pyhd3eb1b0_3
pickleshare 0.7.5 pyhd3eb1b0_1003
pillow 9.2.0 py39hde71d04_1
pinecone-client 2.2.1 pypi_0 pypi
pip 22.1.2 py39hecd8cb5_0
pixman 0.38.0 h01d97ff_1003 conda-forge
platformdirs 2.5.2 py39hecd8cb5_0
prometheus_client 0.14.1 py39hecd8cb5_0
prompt-toolkit 3.0.36 py39hecd8cb5_0
psutil 5.9.0 py39hca72f7f_0
ptyprocess 0.7.0 pyhd3eb1b0_2
pure_eval 0.2.2 pyhd3eb1b0_0
pycparser 2.21 pyhd3eb1b0_0
pydantic 1.10.8 py39hdc70f33_0 conda-forge
pygments 2.11.2 pyhd3eb1b0_0
pygtrie 2.5.0 pypi_0 pypi
pyjwt 2.8.0 pypi_0 pypi
pyopenssl 22.0.0 pyhd3eb1b0_0
pyparsing 3.0.9 py39hecd8cb5_0
pyrsistent 0.18.0 py39hca72f7f_0
pysocks 1.7.1 py39hecd8cb5_0
python 3.9.13 hdfd78df_1
python-dateutil 2.8.2 pyhd3eb1b0_0
python-fastjsonschema 2.16.2 py39hecd8cb5_0
python_abi 3.9 2_cp39 conda-forge
pytorch 1.12.1 cpu_py39h64f2f56_1
pytz 2022.1 py39hecd8cb5_0
pywavelets 1.4.1 pypi_0 pypi
pyyaml 6.0 py39ha30fb19_5 conda-forge
pyzmq 23.2.0 py39he9d5cce_0
qdldl 0.1.7 pypi_0 pypi
qt-main 5.15.2 h719ae48_7
qt-webengine 5.15.9 h90a370e_4
qtwebkit 5.212 h24dc246_4
readline 8.1.2 hca72f7f_1
regex 2023.6.3 pypi_0 pypi
requests 2.28.1 py39hecd8cb5_0
safetensors 0.3.1 pypi_0 pypi
scikit-image 0.19.3 pypi_0 pypi
scikit-learn 1.2.2 pypi_0 pypi
scikit-survival 0.21.0 pypi_0 pypi
scipy 1.8.1 py39h29d19b3_2 conda-forge
send2trash 1.8.0 pyhd3eb1b0_1
sentence-transformers 2.2.2 pypi_0 pypi
sentencepiece 0.1.99 pypi_0 pypi
setuptools 63.4.1 py39hecd8cb5_0
six 1.16.0 pyhd3eb1b0_1
sniffio 1.2.0 py39hecd8cb5_1
soupsieve 2.3.2.post1 py39hecd8cb5_0
sqlalchemy 2.0.15 py39hdc70f33_0 conda-forge
sqlite 3.39.2 h707629a_0
stack_data 0.2.0 pyhd3eb1b0_0
stringcase 1.2.0 py_0 conda-forge
tenacity 8.2.2 pyhd8ed1ab_0 conda-forge
terminado 0.17.1 py39hecd8cb5_0
threadpoolctl 2.2.0 pyh0d69192_0
tifffile 2023.2.3 pypi_0 pypi
tiktoken 0.4.0 pypi_0 pypi
tinycss2 1.2.1 py39hecd8cb5_0
tk 8.6.12 h5d9f67b_0
tokenizers 0.13.3 pypi_0 pypi
torchaudio 0.12.1 py39_cpu pytorch
torchvision 0.13.1 py39_cpu pytorch
tornado 6.2 py39hca72f7f_0
tqdm 4.65.0 pyhd8ed1ab_1 conda-forge
traitlets 5.7.1 py39hecd8cb5_0
transformers 4.30.2 pypi_0 pypi
typing-extensions 4.3.0 py39hecd8cb5_0
typing_extensions 4.3.0 py39hecd8cb5_0
typing_inspect 0.9.0 pyhd8ed1ab_0 conda-forge
tzdata 2022c h04d1e81_0
urllib3 1.26.11 py39hecd8cb5_0
wcwidth 0.2.5 pyhd3eb1b0_0
webencodings 0.5.1 py39hecd8cb5_1
websocket-client 0.58.0 py39hecd8cb5_4
wheel 0.37.1 pyhd3eb1b0_0
xz 5.2.5 hca72f7f_1
yaml 0.2.5 h0d85af4_2 conda-forge
yarl 1.9.2 py39hdc70f33_0 conda-forge
zeromq 4.3.4 h23ab428_0
zipp 3.14.0 pypi_0 pypi
zlib 1.2.12 h4dc903c_3
zstd 1.5.2 hcb37349_0
Is there a known issue when using Guidance with a VPN or any suggestions on how I might resolve this? Thank you in advance for your help. Looking forward to your reply! π₯Ίπ₯Ίπ₯Ίπ₯Ίπ₯Ίπ₯Ί
me too, it solved?
Not yet, I'm still waiting for a solution or workaround.π΅βπ«π΅βπ«π΅βπ«
Hi @Qing1Zhong and @dalong2hongmei,
Thanks for reporting this! I'm not entirely sure what may be causing this -- it is odd that the OpenAI calls naturally run through while the Guidance calls fail. I wonder if it's a streaming related issue?
Do you mind testing this code out?
import openai
openai.api_key = 'My_API_KEY'
response = openai.ChatCompletion.create(
model='gpt-3.5-turbo',
messages=[
{'role': 'user', 'content': "What's 1+1? Answer in one word."}
],
temperature=0,
stream=True # this time, we set stream=True
)
for chunk in response:
print(chunk)
Hi @Harsha-Nori , Thank you for the prompt response!!!πππ I've tested the code you provided, and it runs without any errors. Result:
{
"id": "chatcmpl-7qE02z3mUo1ezoq3VUhPYtDRn3lga",
"object": "chat.completion.chunk",
"created": 1692681226,
"model": "gpt-3.5-turbo-0613",
"choices": [
{
"index": 0,
"delta": {
"role": "assistant",
"content": ""
},
"finish_reason": null
}
]
}
{
"id": "chatcmpl-7qE02z3mUo1ezoq3VUhPYtDRn3lga",
"object": "chat.completion.chunk",
"created": 1692681226,
"model": "gpt-3.5-turbo-0613",
"choices": [
{
"index": 0,
"delta": {
"content": "2"
},
"finish_reason": null
}
]
}
{
"id": "chatcmpl-7qE02z3mUo1ezoq3VUhPYtDRn3lga",
"object": "chat.completion.chunk",
"created": 1692681226,
"model": "gpt-3.5-turbo-0613",
"choices": [
{
"index": 0,
"delta": {},
"finish_reason": "stop"
}
]
}
Do you have any insights on what might be causing this or any further steps I should take? I appreciate your help!
The code:
