dify icon indicating copy to clipboard operation
dify copied to clipboard

docker-plugin_daemon-1 | 2025/03/16 13:18:49 watcher.go:85: [ERROR]plugin launch error: init environment for plugin langgenius/siliconflow:0.0.7 failed too many times, you should consider the package is corrupted or your network is unstable

Open dshwei opened this issue 11 months ago • 1 comments

Self Checks

  • [x] This is only for bug report, if you would like to ask a question, please head to Discussions.
  • [x] I have searched for existing issues search for existing issues, including closed ones.
  • [x] I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
  • [x] [FOR CHINESE USERS] 请务必使用英文提交 Issue,否则会被关闭。谢谢!:)
  • [x] Please do not modify this template :) and fill in all the required fields.

Dify version

1.0.1

Cloud or Self Hosted

Self Hosted (Source), Self Hosted (Docker)

Steps to reproduce

docker-redis-1 | 1:M 16 Mar 2025 13:15:42.199 * Background saving terminated with success docker-plugin_daemon-1 | 2025/03/16 13:15:43 runtime_lifetime.go:76: [ERROR]init environment failed: failed to install dependencies: signal: killed, output: Resolved 41 packages in 2.52s docker-plugin_daemon-1 | Downloading gevent (6.4MiB) docker-plugin_daemon-1 | init process exited due to no activity for 120 seconds, retry in 30s docker-db-1 | 2025-03-16 13:15:47.181 UTC [163] LOG: checkpoint starting: time docker-db-1 | 2025-03-16 13:15:50.835 UTC [163] LOG: checkpoint complete: wrote 30 buffers (0.2%); 0 WAL file(s) added, 0 removed, 0 recycled; write=3.364 s, sync=0.254 s, total=3.655 s; sync files=19, longest=0.080 s, average=0.014 s; distance=11 kB, estimate=2702 kB docker-plugin_daemon-1 | [GIN] 2025/03/16 - 13:16:00 | 200 | 6.039976ms | 172.19.0.7 | GET "/plugin/2a439d3a-c10b-4baf-a701-e57d206df168/management/models?page=1&page_size=256" docker-plugin_daemon-1 | 2025/03/16 13:16:00 factory.go:28: [ERROR]PluginDaemonInternalServerError: no available node, plugin not found docker-plugin_daemon-1 | goroutine 1681 [running]: docker-plugin_daemon-1 | runtime/debug.Stack() docker-plugin_daemon-1 | /usr/local/go/src/runtime/debug/stack.go:24 +0x5e docker-plugin_daemon-1 | github.com/langgenius/dify-plugin-daemon/internal/types/exception.InternalServerError({0x18112a0, 0xc0005ca360}) docker-plugin_daemon-1 | /app/internal/types/exception/factory.go:27 +0x27 docker-plugin_daemon-1 | github.com/langgenius/dify-plugin-daemon/internal/server.(*App).redirectPluginInvokeByPluginIdentifier(0xc0001160c0, 0xc000364600, {0xc000a4e480?, 0xe9d6ff?}, {0x18112a0, 0xc0005ca310}) docker-plugin_daemon-1 | /app/internal/server/middleware.go:120 +0xef docker-plugin_daemon-1 | github.com/langgenius/dify-plugin-daemon/internal/server.(*App).pluginDispatchGroup.(*App).RedirectPluginInvoke.func2(0xc000364600) docker-plugin_daemon-1 | /app/internal/server/middleware.go:94 +0x95 docker-plugin_daemon-1 | github.com/gin-gonic/gin.(*Context).Next(0xc000364600) docker-plugin_daemon-1 | /go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:185 +0x2b docker-plugin_daemon-1 | github.com/langgenius/dify-plugin-daemon/internal/server.(*App).pluginDispatchGroup.(*App).FetchPluginInstallation.func1(0xc000364600) docker-plugin_daemon-1 | /app/internal/server/middleware.go:66 +0x2f2 docker-plugin_daemon-1 | github.com/gin-gonic/gin.(*Context).Next(0xc000364600) docker-plugin_daemon-1 | /go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:185 +0x2b docker-plugin_daemon-1 | github.com/langgenius/dify-plugin-daemon/internal/server.(*App).pluginGroup.CheckingKey.func1(0xc000364600) docker-plugin_daemon-1 | /app/internal/server/middleware.go:24 +0x95 docker-plugin_daemon-1 | github.com/gin-gonic/gin.(*Context).Next(...) docker-plugin_daemon-1 | /go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:185 docker-plugin_daemon-1 | github.com/gin-gonic/gin.CustomRecoveryWithWriter.func1(0xc000364600) docker-plugin_daemon-1 | /go/pkg/mod/github.com/gin-gonic/[email protected]/recovery.go:102 +0x7a docker-plugin_daemon-1 | github.com/gin-gonic/gin.(*Context).Next(...) docker-plugin_daemon-1 | /go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:185 docker-plugin_daemon-1 | github.com/gin-gonic/gin.LoggerWithConfig.func1(0xc000364600) docker-plugin_daemon-1 | /go/pkg/mod/github.com/gin-gonic/[email protected]/logger.go:249 +0xe5 docker-plugin_daemon-1 | github.com/gin-gonic/gin.(*Context).Next(...) docker-plugin_daemon-1 | /go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:185 docker-plugin_daemon-1 | github.com/gin-gonic/gin.(*Engine).handleHTTPRequest(0xc0002aa9c0, 0xc000364600) docker-plugin_daemon-1 | /go/pkg/mod/github.com/gin-gonic/[email protected]/gin.go:633 +0x892 docker-plugin_daemon-1 | github.com/gin-gonic/gin.(*Engine).ServeHTTP(0xc0002aa9c0, {0x1823720, 0xc0005a4000}, 0xc000e40240) docker-plugin_daemon-1 | /go/pkg/mod/github.com/gin-gonic/[email protected]/gin.go:589 +0x1b2 docker-plugin_daemon-1 | net/http.serverHandler.ServeHTTP({0x18194c8?}, {0x1823720?, 0xc0005a4000?}, 0x6?) docker-plugin_daemon-1 | /usr/local/go/src/net/http/server.go:3142 +0x8e docker-plugin_daemon-1 | net/http.(*conn).serve(0xc0005e0000, {0x1825ed0, 0xc0004878f0}) docker-plugin_daemon-1 | /usr/local/go/src/net/http/server.go:2044 +0x5e8 docker-plugin_daemon-1 | created by net/http.(*Server).Serve in goroutine 42 docker-plugin_daemon-1 | /usr/local/go/src/net/http/server.go:3290 +0x4b4 docker-plugin_daemon-1 | docker-plugin_daemon-1 | [GIN] 2025/03/16 - 13:16:00 | 404 | 5.153097ms | 172.19.0.7 | POST "/plugin/2a439d3a-c10b-4baf-a701-e57d206df168/dispatch/model/validate_provider_credentials" docker-nginx-1 | 172.19.0.1 - - [16/Mar/2025:13:16:00 +0000] "POST /console/api/workspaces/current/model-providers/langgenius/tongyi/tongyi HTTP/1.1" 500 56 "http://localhost:9090/app/e7de5c83-1f0b-434c-b42e-621b1d3df261/configuration" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/133.0.0.0 Safari/537.36 Edg/133.0.0.0" "-" docker-api-1 | 2025-03-16 13:16:00.364 ERROR [Dummy-20] [app.py:875] - Exception on /console/api/workspaces/current/model-providers/langgenius/tongyi/tongyi [POST] docker-api-1 | Traceback (most recent call last): docker-api-1 | File "/app/api/.venv/lib/python3.12/site-packages/flask/app.py", line 917, in full_dispatch_request docker-api-1 | rv = self.dispatch_request() docker-api-1 | ^^^^^^^^^^^^^^^^^^^^^^^ docker-api-1 | File "/app/api/.venv/lib/python3.12/site-packages/flask/app.py", line 902, in dispatch_request docker-api-1 | return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) # type: ignore[no-any-return] docker-api-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ docker-api-1 | File "/app/api/.venv/lib/python3.12/site-packages/flask_restful/init.py", line 489, in wrapper docker-api-1 | resp = resource(*args, **kwargs) docker-api-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^ docker-api-1 | File "/app/api/.venv/lib/python3.12/site-packages/flask/views.py", line 110, in view docker-api-1 | return current_app.ensure_sync(self.dispatch_request)(**kwargs) # type: ignore[no-any-return] docker-api-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ docker-api-1 | File "/app/api/.venv/lib/python3.12/site-packages/flask_restful/init.py", line 604, in dispatch_request docker-api-1 | resp = meth(*args, **kwargs) docker-api-1 | ^^^^^^^^^^^^^^^^^^^^^ docker-api-1 | File "/app/api/controllers/console/wraps.py", line 187, in decorated docker-api-1 | return view(*args, **kwargs) docker-api-1 | ^^^^^^^^^^^^^^^^^^^^^ docker-api-1 | File "/app/api/libs/login.py", line 94, in decorated_view docker-api-1 | return current_app.ensure_sync(func)(*args, **kwargs) docker-api-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ docker-api-1 | File "/app/api/controllers/console/wraps.py", line 30, in decorated docker-api-1 | return view(*args, **kwargs) docker-api-1 | ^^^^^^^^^^^^^^^^^^^^^ docker-api-1 | File "/app/api/controllers/console/workspace/model_providers.py", line 102, in post docker-api-1 | model_provider_service.save_provider_credentials( docker-api-1 | File "/app/api/services/model_provider_service.py", line 145, in save_provider_credentials docker-api-1 | provider_configuration.add_or_update_custom_credentials(credentials) docker-api-1 | File "/app/api/core/entities/provider_configuration.py", line 257, in add_or_update_custom_credentials docker-api-1 | provider_record, credentials = self.custom_credentials_validate(credentials) docker-api-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ docker-api-1 | File "/app/api/core/entities/provider_configuration.py", line 240, in custom_credentials_validate docker-api-1 | credentials = model_provider_factory.provider_credentials_validate( docker-api-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ docker-api-1 | File "/app/api/core/model_runtime/model_providers/model_provider_factory.py", line 156, in provider_credentials_validate docker-api-1 | self.plugin_model_manager.validate_provider_credentials( docker-api-1 | File "/app/api/core/plugin/manager/model.py", line 97, in validate_provider_credentials docker-api-1 | for resp in response: docker-api-1 | ^^^^^^^^ docker-api-1 | File "/app/api/core/plugin/manager/base.py", line 189, in _request_with_plugin_daemon_response_stream docker-api-1 | self._handle_plugin_daemon_error(error.error_type, error.message) docker-api-1 | File "/app/api/core/plugin/manager/base.py", line 223, in _handle_plugin_daemon_error docker-api-1 | raise PluginDaemonInternalServerError(description=message) docker-api-1 | core.plugin.manager.exc.PluginDaemonInternalServerError: PluginDaemonInternalServerError: no available node, plugin not found docker-sandbox-1 | [GIN] 2025/03/16 - 13:16:01 | 200 | 35.32µs | 127.0.0.1 | GET "/health" docker-plugin_daemon-1 | 2025/03/16 13:16:13 runtime_lifetime.go:70: [INFO]init environment for plugin langgenius/siliconflow:0.0.7 docker-sandbox-1 | [GIN] 2025/03/16 - 13:16:31 | 200 | 15.301µs | 127.0.0.1 | GET "/health" docker-plugin_daemon-1 | 2025/03/16 13:16:38 runtime_lifetime.go:76: [ERROR]init environment failed: failed to install dependencies: signal: killed, output: Resolved 37 packages in 1.60s docker-plugin_daemon-1 | Downloading gevent (6.4MiB) docker-plugin_daemon-1 | init process exited due to no activity for 120 seconds, retry in 30s docker-plugin_daemon-1 | 2025/03/16 13:16:53 factory.go:28: [ERROR]PluginDaemonInternalServerError: killed by timeout docker-plugin_daemon-1 | goroutine 1005 [running]: docker-plugin_daemon-1 | runtime/debug.Stack() docker-plugin_daemon-1 | /usr/local/go/src/runtime/debug/stack.go:24 +0x5e docker-plugin_daemon-1 | github.com/langgenius/dify-plugin-daemon/internal/types/exception.InternalServerError({0x18112a0, 0xc000f97550}) docker-plugin_daemon-1 | /app/internal/types/exception/factory.go:27 +0x27 docker-plugin_daemon-1 | github.com/langgenius/dify-plugin-daemon/internal/service.baseSSEService[...](0xc000e5f0e0, 0x15fbb1c, 0x258?) docker-plugin_daemon-1 | /app/internal/service/base_sse.go:81 +0x4e6 docker-plugin_daemon-1 | github.com/langgenius/dify-plugin-daemon/internal/service.ValidateProviderCredentials(0xc000464500, 0xc000364100, 0x258) docker-plugin_daemon-1 | /app/internal/service/invoke_model.go:208 +0x198 docker-plugin_daemon-1 | github.com/langgenius/dify-plugin-daemon/internal/server.(*App).pluginDispatchGroup.ValidateProviderCredentials.func17.1({{{0xc0007c600d, 0x24}, {0xc0007b4260, 0x7}}, {{0x0, 0x0}}, {0xc000bd8120, 0x5a}, 0x0, 0x0, ...}) docker-plugin_daemon-1 | /app/internal/server/controllers/model.go:98 +0x91 docker-plugin_daemon-1 | github.com/langgenius/dify-plugin-daemon/internal/server/controllers.BindPluginDispatchRequest[...].func1() docker-plugin_daemon-1 | /app/internal/server/controllers/base.go:53 +0xb5 docker-plugin_daemon-1 | github.com/langgenius/dify-plugin-daemon/internal/server/controllers.BindRequest[...](0xc000364100, 0xc000e5f328) docker-plugin_daemon-1 | /app/internal/server/controllers/base.go:31 +0x275 docker-plugin_daemon-1 | github.com/langgenius/dify-plugin-daemon/internal/server/controllers.BindPluginDispatchRequest[...](0x160893f?, 0xa?) docker-plugin_daemon-1 | /app/internal/server/controllers/base.go:37 +0x45 docker-plugin_daemon-1 | github.com/langgenius/dify-plugin-daemon/internal/server.(*App).pluginDispatchGroup.ValidateProviderCredentials.func17(0xc000364100) docker-plugin_daemon-1 | /app/internal/server/controllers/model.go:95 +0x74 docker-plugin_daemon-1 | github.com/gin-gonic/gin.(*Context).Next(0xc000364100) docker-plugin_daemon-1 | /go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:185 +0x2b docker-plugin_daemon-1 | github.com/langgenius/dify-plugin-daemon/internal/server.(*App).pluginDispatchGroup.(*App).InitClusterID.func3(0xc000364100) docker-plugin_daemon-1 | /app/internal/server/middleware.go:170 +0x54 docker-plugin_daemon-1 | github.com/gin-gonic/gin.(*Context).Next(0xc000364100) docker-plugin_daemon-1 | /go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:185 +0x2b docker-plugin_daemon-1 | github.com/langgenius/dify-plugin-daemon/internal/server.(*App).pluginDispatchGroup.(*App).RedirectPluginInvoke.func2(0xc000364100) docker-plugin_daemon-1 | /app/internal/server/middleware.go:97 +0x74 docker-plugin_daemon-1 | github.com/gin-gonic/gin.(*Context).Next(0xc000364100) docker-plugin_daemon-1 | /go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:185 +0x2b docker-plugin_daemon-1 | github.com/langgenius/dify-plugin-daemon/internal/server.(*App).pluginDispatchGroup.(*App).FetchPluginInstallation.func1(0xc000364100) docker-plugin_daemon-1 | /app/internal/server/middleware.go:66 +0x2f2 docker-plugin_daemon-1 | github.com/gin-gonic/gin.(*Context).Next(0xc000364100) docker-plugin_daemon-1 | /go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:185 +0x2b docker-plugin_daemon-1 | github.com/langgenius/dify-plugin-daemon/internal/server.(*App).pluginGroup.CheckingKey.func1(0xc000364100) docker-plugin_daemon-1 | /app/internal/server/middleware.go:24 +0x95 docker-plugin_daemon-1 | github.com/gin-gonic/gin.(*Context).Next(...) docker-plugin_daemon-1 | /go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:185 docker-plugin_daemon-1 | github.com/gin-gonic/gin.CustomRecoveryWithWriter.func1(0xc000364100) docker-plugin_daemon-1 | /go/pkg/mod/github.com/gin-gonic/[email protected]/recovery.go:102 +0x7a docker-plugin_daemon-1 | github.com/gin-gonic/gin.(*Context).Next(...) docker-plugin_daemon-1 | /go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:185 docker-plugin_daemon-1 | github.com/gin-gonic/gin.LoggerWithConfig.func1(0xc000364100) docker-plugin_daemon-1 | /go/pkg/mod/github.com/gin-gonic/[email protected]/logger.go:249 +0xe5 docker-plugin_daemon-1 | github.com/gin-gonic/gin.(*Context).Next(...) docker-plugin_daemon-1 | /go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:185 docker-plugin_daemon-1 | github.com/gin-gonic/gin.(*Engine).handleHTTPRequest(0xc0002aa9c0, 0xc000364100) docker-plugin_daemon-1 | /go/pkg/mod/github.com/gin-gonic/[email protected]/gin.go:633 +0x892 docker-plugin_daemon-1 | github.com/gin-gonic/gin.(*Engine).ServeHTTP(0xc0002aa9c0, {0x1823720, 0xc0001be1c0}, 0xc00092c000) docker-plugin_daemon-1 | /go/pkg/mod/github.com/gin-gonic/[email protected]/gin.go:589 +0x1b2 docker-plugin_daemon-1 | net/http.serverHandler.ServeHTTP({0x18194c8?}, {0x1823720?, 0xc0001be1c0?}, 0x6?) docker-plugin_daemon-1 | /usr/local/go/src/net/http/server.go:3142 +0x8e docker-plugin_daemon-1 | net/http.(*conn).serve(0xc00059c090, {0x1825ed0, 0xc0004878f0}) docker-plugin_daemon-1 | /usr/local/go/src/net/http/server.go:2044 +0x5e8 docker-plugin_daemon-1 | created by net/http.(*Server).Serve in goroutine 42 docker-plugin_daemon-1 | /usr/local/go/src/net/http/server.go:3290 +0x4b4 docker-plugin_daemon-1 | docker-plugin_daemon-1 | [GIN] 2025/03/16 - 13:16:53 | 200 | 10m0s | 172.19.0.7 | POST "/plugin/2a439d3a-c10b-4baf-a701-e57d206df168/dispatch/model/validate_provider_credentials" docker-api-1 | 2025-03-16 13:16:53.265 ERROR [Dummy-21] [app.py:875] - Exception on /console/api/workspaces/current/model-providers/langgenius/deepseek/deepseek [POST]

✔️ Expected Behavior

Problems that arise when configuring api-key after installing the model vendor

❌ Actual Behavior

To normal api-key can be configured

dshwei avatar Mar 16 '25 13:03 dshwei

To resolve the "PluginDaemonInternalServerError: no available node, plugin not found" error when configuring the API key for the langgenius/siliconflow:0.0.7 plugin in a self-hosted Docker environment, you can try the following solutions:

  1. Increase the Python Environment Initialization Timeout: Add the following line to your docker-compose.yaml file under the plugin_daemon service to increase the timeout:

    PYTHON_ENV_INIT_TIMEOUT: 320
    

    This helps prevent the error if the plugin fails to initialize within the default timeout period.

  2. Clear Plugin Data and Directories: If the above solution does not work, perform these additional steps:

    • Clear plugin data from the database using specific SQL commands.
    • Clear plugin directories from:
      • /app/storage/cwd/plugins/plugin
      • /app/storage/cwd/plugins/plugin-working
      • /app/storage/cwd/plugins/persistence
      • /app/storage/cwd/plugins/plugin-root
    • Restart the plugin daemon service.
    • Reinstall the plugins.
  3. Use a PIP Mirror URL: If you are using an internal network, consider adding the environment variable PIP_MIRROR_URL to the plugin_daemon service with a mirror URL, such as:

    PIP_MIRROR_URL: https://pypi.tuna.tsinghua.edu.cn/simple
    

These steps address common issues related to plugin installation failures due to Python dependencies not being installed within the default timeout period [1].

To continue talking to Dosu, mention @dosu.


Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other

dosubot[bot] avatar Mar 16 '25 13:03 dosubot[bot]

I noticed that you have start the application from both source code and docker, if you are adding the same provider to each of them, then the data will be messed up. In this case, you need to clean the database to remove this record or purge the database and perform a fresh install again.

crazywoola avatar Mar 17 '25 01:03 crazywoola

hi, every body, I make it! I use helm based k8s, as below: pluginDaemon: repository: langgenius/dify-plugin-daemon tag: 0.0.3-local pullPolicy: IfNotPresent extraEnv: - name: PIP_MIRROR_URL value: https://pypi.tuna.tsinghua.edu.cn/simple

zhangruilin2020 avatar Apr 12 '25 09:04 zhangruilin2020

I got the same problem with my dify service running on an arm64 offline server, and i am preparing to build up a offline pip mirror source for plugin-daemon's PIP_MIRROR_URL, if anyone has offline pip mirror docker image or any other way to run and manage plugins' dependences, please let me know, thx.

Image

Amthurson avatar Apr 29 '25 08:04 Amthurson

tsinghua failed, try Ali: https://mirrors.aliyun.com/pypi/simple

liaoweiguo avatar Jun 27 '25 02:06 liaoweiguo

where to config PIP_MIRROR_URL?

fanfan-lucky avatar Jul 01 '25 08:07 fanfan-lucky