flyte icon indicating copy to clipboard operation
flyte copied to clipboard
verified publisher icon flyteorg

[BUG] Flytekit Auth via flytekit on Windows not being supported

Open fiedlerNr9 opened this issue 1 year ago • 4 comments

Describe the bug

Authenticating against an auth enabled flyte deployment fails on windows at this moment:

pyflyte -c config.yaml get launchplan

Failed with Exception Code: SYSTEM:Unknown
Traceback:
  File "C:\Users\jan\AppData\Local\Programs\Python\Python312\Lib\site-packages\grpc\_interceptor.py", line 315, in continuation
    response, call = self._thunk(new_method).with_call(
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "C:\Users\jan\AppData\Local\Programs\Python\Python312\Lib\site-packages\grpc\_interceptor.py", line 343, in with_call
    return self._with_call(
           ^^^^^^^^^^^^^^^^

  File "C:\Users\jan\AppData\Local\Programs\Python\Python312\Lib\site-packages\grpc\_interceptor.py", line 329, in _with_call
    call = self._interceptor.intercept_unary_unary(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "C:\Users\jan\AppData\Local\Programs\Python\Python312\Lib\site-packages\flytekit\clients\grpc_utils\auth_interceptor.py", line 65, in intercept_unary_unary
    self._authenticator.refresh_credentials()

  File "C:\Users\jan\AppData\Local\Programs\Python\Python312\Lib\site-packages\flytekit\clients\auth\authenticator.py", line 164, in refresh_credentials
    self._creds = self._auth_client.get_creds_from_remote()
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "C:\Users\jan\AppData\Local\Programs\Python\Python312\Lib\site-packages\flytekit\clients\auth\auth_client.py", line 348, in get_creds_from_remote
    ctx = get_context("fork")
          ^^^^^^^^^^^^^^^^^^^

  File "C:\Users\jan\AppData\Local\Programs\Python\Python312\Lib\multiprocessing\context.py", line 243, in get_context
    return super().get_context(method)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "C:\Users\jan\AppData\Local\Programs\Python\Python312\Lib\multiprocessing\context.py", line 193, in get_context
    raise ValueError('cannot find context for %r' % method) from None

This fails because in the auth_client we use the fork multiprocessing method, which is not supported for Windows. More Context below!

Expected behavior

Flytekit should open up a Browser Window to authenticate.

Additional context to reproduce

This fails because in the auth_client we use the fork multiprocessing method. This Line.

According these Docs fork is not supported for Windows. We would need to use spawn.

I tried replacing with spawn and got the following errors on mac os & Windows.

MAC:

pyflyte -c config.yaml get launchplan                                                                                                                                                    
WARNING:root:Failed to refresh token. Kicking off a full authorization flow.
WARNING:root:Failed to refresh token. Kicking off a full authorization flow.
WARNING:root:Failed to refresh token. Kicking off a full authorization flow.
WARNING:root:Failed to refresh token. Kicking off a full authorization flow.
Failed with Exception Code: SYSTEM:Unknown
Traceback:
  File "/opt/homebrew/anaconda3/envs/windows_auth/lib/python3.9/site-packages/grpc/_interceptor.py", line 315, in continuation
    response, call = self._thunk(new_method).with_call(

  File "/opt/homebrew/anaconda3/envs/windows_auth/lib/python3.9/site-packages/grpc/_interceptor.py", line 343, in with_call
    return self._with_call(

  File "/opt/homebrew/anaconda3/envs/windows_auth/lib/python3.9/site-packages/grpc/_interceptor.py", line 329, in _with_call
    call = self._interceptor.intercept_unary_unary(

  File "/Users/janfiedler/Documents/Flyte/flytekit/flytekit/clients/grpc_utils/auth_interceptor.py", line 65, in intercept_unary_unary
    self._authenticator.refresh_credentials()

  File "/Users/janfiedler/Documents/Flyte/flytekit/flytekit/clients/auth/authenticator.py", line 164, in refresh_credentials
    self._creds = self._auth_client.get_creds_from_remote()

  File "/Users/janfiedler/Documents/Flyte/flytekit/flytekit/clients/auth/auth_client.py", line 377, in get_creds_from_remote
    server = self._create_callback_server()

  File "/Users/janfiedler/Documents/Flyte/flytekit/flytekit/clients/auth/auth_client.py", line 288, in _create_callback_server
    return OAuthHTTPServer(

  File "/Users/janfiedler/Documents/Flyte/flytekit/flytekit/clients/auth/auth_client.py", line 138, in __init__
    _BaseHTTPServer.HTTPServer.__init__(

  File "/opt/homebrew/anaconda3/envs/windows_auth/lib/python3.9/socketserver.py", line 452, in __init__
    self.server_bind()

  File "/opt/homebrew/anaconda3/envs/windows_auth/lib/python3.9/http/server.py", line 137, in server_bind
    socketserver.TCPServer.server_bind(self)

  File "/opt/homebrew/anaconda3/envs/windows_auth/lib/python3.9/socketserver.py", line 466, in server_bind
    self.socket.bind(self.server_address)

Windows:

(windows_auth) PS C:\Users\jan\Documents\union\union> pyflyte -c .\config.yaml get launchplan
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Users\jan\miniconda3\envs\windows_auth\lib\multiprocessing\spawn.py", line 116, in spawn_main
    exitcode = _main(fd, parent_sentinel)
  File "C:\Users\jan\miniconda3\envs\windows_auth\lib\multiprocessing\spawn.py", line 126, in _main
    self = reduction.pickle.load(from_parent)
EOFError: Ran out of input
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Users\jan\miniconda3\envs\windows_auth\lib\multiprocessing\spawn.py", line 116, in spawn_main
    exitcode = _main(fd, parent_sentinel)
  File "C:\Users\jan\miniconda3\envs\windows_auth\lib\multiprocessing\spawn.py", line 126, in _main
    self = reduction.pickle.load(from_parent)
EOFError: Ran out of input
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Users\jan\miniconda3\envs\windows_auth\lib\multiprocessing\spawn.py", line 116, in spawn_main
    exitcode = _main(fd, parent_sentinel)
  File "C:\Users\jan\miniconda3\envs\windows_auth\lib\multiprocessing\spawn.py", line 126, in _main
    self = reduction.pickle.load(from_parent)
EOFError: Ran out of input
Failed with Exception Code: SYSTEM:Unknown
Traceback:
  File "C:\Users\jan\miniconda3\envs\windows_auth\lib\site-packages\grpc\_interceptor.py", line 315, in continuation
    response, call = self._thunk(new_method).with_call(

  File "C:\Users\jan\miniconda3\envs\windows_auth\lib\site-packages\grpc\_interceptor.py", line 343, in with_call
    return self._with_call(

  File "C:\Users\jan\miniconda3\envs\windows_auth\lib\site-packages\grpc\_interceptor.py", line 329, in _with_call
    call = self._interceptor.intercept_unary_unary(

  File "C:\Users\jan\Documents\union\union\flytekit\flytekit\clients\grpc_utils\auth_interceptor.py", line 65, in intercept_unary_unary
    self._authenticator.refresh_credentials()

  File "C:\Users\jan\Documents\union\union\flytekit\flytekit\clients\auth\authenticator.py", line 164, in refresh_credentials
    self._creds = self._auth_client.get_creds_from_remote()

  File "C:\Users\jan\Documents\union\union\flytekit\flytekit\clients\auth\auth_client.py", line 392, in get_creds_from_remote
    server_process.terminate()

  File "C:\Users\jan\miniconda3\envs\windows_auth\lib\multiprocessing\process.py", line 133, in terminate
    self._popen.terminate()


Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Users\jan\miniconda3\envs\windows_auth\lib\multiprocessing\spawn.py", line 116, in spawn_main
    exitcode = _main(fd, parent_sentinel)
  File "C:\Users\jan\miniconda3\envs\windows_auth\lib\multiprocessing\spawn.py", line 126, in _main
    self = reduction.pickle.load(from_parent)
EOFError: Ran out of input

I think we should stay with fork for every platform except Windows. The Error for Windows looks like its coming from the nature of spawn since not all resources are shared from the parent process?

Screenshots

No response

Are you sure this issue hasn't been raised already?

  • [X] Yes

Have you read the Code of Conduct?

  • [X] Yes

fiedlerNr9 avatar Feb 29 '24 12:02 fiedlerNr9

Thank you for opening your first issue here! 🛠

welcome[bot] avatar Feb 29 '24 12:02 welcome[bot]

🤖

It's great to see you've identified the issue with authenticating against an auth-enabled Flyte deployment on Windows. Your detailed analysis and attempts to resolve the issue are really helpful. If you're able to propose a solution or fix for this, we'd encourage you to create a pull request with your changes. Your contribution would be much appreciated!

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

dosubot[bot] avatar Feb 29 '24 12:02 dosubot[bot]

Related to https://github.com/flyteorg/flyte/issues/5079

wild-endeavor avatar Apr 23 '24 23:04 wild-endeavor

Related to #5079

I posted a workaround we've been using in this Issue. We haven't seen any bugs from it yet but I wouldn't want to use this monkeypatch long term.

ai-rnatour avatar Apr 26 '24 18:04 ai-rnatour