Shuffle icon indicating copy to clipboard operation
Shuffle copied to clipboard
verified publisher icon Shuffle

SHUFFLE_DEFAULT_USERNAME doesn't set up account

Open wilson-conley opened this issue 1 year ago • 1 comments

Describe the bug I've set the SHUFFLE_DEFAULT_USERNAME, SHUFFLE_DEFAULT_PASSWORD, and SHUFFLE_DEFAULT_APIKEY environment variables in .env, but they don't actually create the default account when I start up Shuffle for the first time.

I've also explicitly set them as environment variables for the backend docker container in docker-compose.yml:

  backend:
  ...
    environment:
      - SHUFFLE_DEFAULT_USERNAME=${SHUFFLE_DEFAULT_USERNAME}
      - SHUFFLE_DEFAULT_PASSWORD=${SHUFFLE_DEFAULT_PASSWORD}
      - SHUFFLE_DEFAULT_APIKEY=${SHUFFLE_DEFAULT_APIKEY}

When I start it up, the backend container attempts to create the account but runs into the following error:

[WARNING] Failed setting organization when creating original user: Not allowed to update an org without any users in the organization. Add at least one user to update

Which I believe comes from:

https://github.com/Shuffle/Shuffle/blob/06ec47a97496440452bcd243ac35dc97c9a0fc5c/backend/go-app/main.go#L3903

To Reproduce Steps to reproduce the behavior:

  1. Set the SHUFFLE_DEFAULT_USERNAME, SHUFFLE_DEFAULT_PASSWORD, and SHUFFLE_DEFAULT_APIKEY environment variables in .env
  2. Set those same environment variables under environment: for the backend container in the docker-compose.yml file
  3. Run docker compose up
  4. On first login to Shuffle webpage, I am still met with the same "Create initial admin account page" and the user credentials I specified in the .env file don't exist on the server ...

Expected behavior Setting the above variables should create a default account to use without having to go through the "first login" prompts in Shuffle.

** Debug logs (NOT APPLICABLE FOR CLOUD)**

2024/10/01 23:10:20 [DEBUG] Inside Files Init with org bucket name "shuffle_org_files"
2024/10/01 23:10:20 [DEBUG] Starting Shuffle backend - initializing database connection
2024/10/01 23:10:20 [DEBUG] Starting with memcached address '' (SHUFFLE_MEMCACHED). If this is empty, fallback to default (appengine / local). Name: 'onprem'
2024/10/01 23:10:20 [DEBUG] Using custom opensearch url 'https://shuffle-opensearch:9200'
2024/10/01 23:10:21 [WARNING] Failed setting up Opensearch: dial tcp 172.18.0.4:9200: connect: connection refused. Typically means the backend can't connect, or that there's a HTTPS vs HTTP problem. Is the SHUFFLE_OPENSEARCH_URL correct?
2024/10/01 23:10:21 [ERROR] Error in initial database connection. Retrying in 5 seconds. dial tcp 172.18.0.4:9200: connect: connection refused
2024/10/01 23:10:26 [DEBUG] Starting with memcached address '' (SHUFFLE_MEMCACHED). If this is empty, fallback to default (appengine / local). Name: 'onprem'
2024/10/01 23:10:26 [DEBUG] Using custom opensearch url 'https://shuffle-opensearch:9200'
2024/10/01 23:10:26 [WARNING] Failed setting up Opensearch: dial tcp 172.18.0.4:9200: connect: connection refused. Typically means the backend can't connect, or that there's a HTTPS vs HTTP problem. Is the SHUFFLE_OPENSEARCH_URL correct?
2024/10/01 23:10:26 [ERROR] Error in initial database connection. Retrying in 5 seconds. dial tcp 172.18.0.4:9200: connect: connection refused
2024/10/01 23:10:31 [DEBUG] Starting with memcached address '' (SHUFFLE_MEMCACHED). If this is empty, fallback to default (appengine / local). Name: 'onprem'
2024/10/01 23:10:31 [DEBUG] Using custom opensearch url 'https://shuffle-opensearch:9200'
2024/10/01 23:10:31 [WARNING] Failed setting up Opensearch: cannot retrieve information from OpenSearch. Typically means the backend can't connect, or that there's a HTTPS vs HTTP problem. Is the SHUFFLE_OPENSEARCH_URL correct?
2024/10/01 23:10:31 [ERROR] Error in initial database connection. Retrying in 5 seconds. cannot retrieve information from OpenSearch
2024/10/01 23:10:36 [DEBUG] Starting with memcached address '' (SHUFFLE_MEMCACHED). If this is empty, fallback to default (appengine / local). Name: 'onprem'
2024/10/01 23:10:36 [DEBUG] Using custom opensearch url 'https://shuffle-opensearch:9200'
2024/10/01 23:10:36 [DEBUG] Initialized Shuffle database connection. Setting up environment.
2024/10/01 23:10:46 [DEBUG] Starting INIT setup for Elasticsearch/Opensearch
2024/10/01 23:10:46 [DEBUG] Getting organizations for Elasticsearch/Opensearch
2024/10/01 23:10:46 [DEBUG] Running on shuffle-backend:5001
2024/10/01 23:10:46 [DEBUG] Got 0 organizations to look into. If this is 0, we wait 10 more seconds until DB is ready and try again.
2024/10/01 23:10:56 [DEBUG] No orgs. Setting NEW org "default"
2024/10/01 23:10:58 [INFO] Cloud action executed successfully for 'init'
2024/10/01 23:10:58 [INFO] Ran initial setup!
2024/10/01 23:10:58 [INFO] Waiting during init to make sure the opensearch instance is up and running with security features properly
2024/10/01 23:11:28 [INFO] Setting up 0 schedule(s)
2024/10/01 23:11:28 [INFO] Trying to set up user based on environments SHUFFLE_DEFAULT_USERNAME & SHUFFLE_DEFAULT_PASSWORD
2024/10/01 23:11:28 [DEBUG] Creating org for default user admin
2024/10/01 23:11:28 [DEBUG] Getting init tutorials for org default (87ec9053-a0e8-4373-8b9b-1e99219a13f9)
2024/10/01 23:11:28 [WARNING] Failed setting organization when creating original user: Not allowed to update an org without any users in the organization. Add at least one user to update
2024/10/01 23:11:28 [INFO] Starting cloud schedules for orgs if enabled!
2024/10/01 23:11:28 [DEBUG] Starting schedule setup for execution cleanup every 300 seconds. Running first immediately. Environments: []string{"Shuffle"}
2024/10/01 23:11:28 [INFO] Running schedule for cleaning up or re-running unfinished workflows in 1 environments.
2024/10/01 23:11:28 [INFO] Getting and validating workflowapps. Got 0 with err <nil>
2024/10/01 23:11:28 [DEBUG] Downloading default apps as 0 were found
2024/10/01 23:11:28 [DEBUG] Getting apps from url 'https://github.com/shuffle/python-apps'
2024/10/01 23:11:28 [WARNING] Api authentication failed in ABORT dangling executions: Couldn't find the user
2024/10/01 23:11:28 [DEBUG] Successfully ran workflow cleanup request for Shuffle. Body: {"success": false}
2024/10/01 23:11:28 [WARNING] Api authentication failed in Rerun executions: Couldn't find the user
2024/10/01 23:11:28 [DEBUG] Successfully ran workflow RERUN request for Shuffle. Body: {"success": false}
2024/10/01 23:11:32 Skipping auth setup: get_excel_file_data
2024/10/01 23:11:33 [WARNING] Skipping build of Yara to later
2024/10/01 23:11:33 [INFO] Starting build of 143 containers (FIRST)
2024/10/01 23:11:33 [INFO] Setting up memory build structure for folder: email/1.1.0/
2024/10/01 23:11:33 [INFO] Building email/1.1.0/ with proxy "". Tags: "frikky/shuffle:email_1.1.0". This may take up to a few minutes.
...

wilson-conley avatar Oct 01 '24 23:10 wilson-conley

Describe the bug I've set the SHUFFLE_DEFAULT_USERNAME, SHUFFLE_DEFAULT_PASSWORD, and SHUFFLE_DEFAULT_APIKEY environment variables in .env, but they don't actually create the default account when I start up Shuffle for the first time.

I've also explicitly set them as environment variables for the backend docker container in docker-compose.yml:

  backend:
  ...
    environment:
      - SHUFFLE_DEFAULT_USERNAME=${SHUFFLE_DEFAULT_USERNAME}
      - SHUFFLE_DEFAULT_PASSWORD=${SHUFFLE_DEFAULT_PASSWORD}
      - SHUFFLE_DEFAULT_APIKEY=${SHUFFLE_DEFAULT_APIKEY}

When I start it up, the backend container attempts to create the account but runs into the following error:

[WARNING] Failed setting organization when creating original user: Not allowed to update an org without any users in the organization. Add at least one user to update

Which I believe comes from:

https://github.com/Shuffle/Shuffle/blob/06ec47a97496440452bcd243ac35dc97c9a0fc5c/backend/go-app/main.go#L3903

To Reproduce Steps to reproduce the behavior:

  1. Set the SHUFFLE_DEFAULT_USERNAME, SHUFFLE_DEFAULT_PASSWORD, and SHUFFLE_DEFAULT_APIKEY environment variables in .env
  2. Set those same environment variables under environment: for the backend container in the docker-compose.yml file
  3. Run docker compose up
  4. On first login to Shuffle webpage, I am still met with the same "Create initial admin account page" and the user credentials I specified in the .env file don't exist on the server ...

Expected behavior Setting the above variables should create a default account to use without having to go through the "first login" prompts in Shuffle.

** Debug logs (NOT APPLICABLE FOR CLOUD)**

2024/10/01 23:10:20 [DEBUG] Inside Files Init with org bucket name "shuffle_org_files"
2024/10/01 23:10:20 [DEBUG] Starting Shuffle backend - initializing database connection
2024/10/01 23:10:20 [DEBUG] Starting with memcached address '' (SHUFFLE_MEMCACHED). If this is empty, fallback to default (appengine / local). Name: 'onprem'
2024/10/01 23:10:20 [DEBUG] Using custom opensearch url 'https://shuffle-opensearch:9200'
2024/10/01 23:10:21 [WARNING] Failed setting up Opensearch: dial tcp 172.18.0.4:9200: connect: connection refused. Typically means the backend can't connect, or that there's a HTTPS vs HTTP problem. Is the SHUFFLE_OPENSEARCH_URL correct?
2024/10/01 23:10:21 [ERROR] Error in initial database connection. Retrying in 5 seconds. dial tcp 172.18.0.4:9200: connect: connection refused
2024/10/01 23:10:26 [DEBUG] Starting with memcached address '' (SHUFFLE_MEMCACHED). If this is empty, fallback to default (appengine / local). Name: 'onprem'
2024/10/01 23:10:26 [DEBUG] Using custom opensearch url 'https://shuffle-opensearch:9200'
2024/10/01 23:10:26 [WARNING] Failed setting up Opensearch: dial tcp 172.18.0.4:9200: connect: connection refused. Typically means the backend can't connect, or that there's a HTTPS vs HTTP problem. Is the SHUFFLE_OPENSEARCH_URL correct?
2024/10/01 23:10:26 [ERROR] Error in initial database connection. Retrying in 5 seconds. dial tcp 172.18.0.4:9200: connect: connection refused
2024/10/01 23:10:31 [DEBUG] Starting with memcached address '' (SHUFFLE_MEMCACHED). If this is empty, fallback to default (appengine / local). Name: 'onprem'
2024/10/01 23:10:31 [DEBUG] Using custom opensearch url 'https://shuffle-opensearch:9200'
2024/10/01 23:10:31 [WARNING] Failed setting up Opensearch: cannot retrieve information from OpenSearch. Typically means the backend can't connect, or that there's a HTTPS vs HTTP problem. Is the SHUFFLE_OPENSEARCH_URL correct?
2024/10/01 23:10:31 [ERROR] Error in initial database connection. Retrying in 5 seconds. cannot retrieve information from OpenSearch
2024/10/01 23:10:36 [DEBUG] Starting with memcached address '' (SHUFFLE_MEMCACHED). If this is empty, fallback to default (appengine / local). Name: 'onprem'
2024/10/01 23:10:36 [DEBUG] Using custom opensearch url 'https://shuffle-opensearch:9200'
2024/10/01 23:10:36 [DEBUG] Initialized Shuffle database connection. Setting up environment.
2024/10/01 23:10:46 [DEBUG] Starting INIT setup for Elasticsearch/Opensearch
2024/10/01 23:10:46 [DEBUG] Getting organizations for Elasticsearch/Opensearch
2024/10/01 23:10:46 [DEBUG] Running on shuffle-backend:5001
2024/10/01 23:10:46 [DEBUG] Got 0 organizations to look into. If this is 0, we wait 10 more seconds until DB is ready and try again.
2024/10/01 23:10:56 [DEBUG] No orgs. Setting NEW org "default"
2024/10/01 23:10:58 [INFO] Cloud action executed successfully for 'init'
2024/10/01 23:10:58 [INFO] Ran initial setup!
2024/10/01 23:10:58 [INFO] Waiting during init to make sure the opensearch instance is up and running with security features properly
2024/10/01 23:11:28 [INFO] Setting up 0 schedule(s)
2024/10/01 23:11:28 [INFO] Trying to set up user based on environments SHUFFLE_DEFAULT_USERNAME & SHUFFLE_DEFAULT_PASSWORD
2024/10/01 23:11:28 [DEBUG] Creating org for default user admin
2024/10/01 23:11:28 [DEBUG] Getting init tutorials for org default (87ec9053-a0e8-4373-8b9b-1e99219a13f9)
2024/10/01 23:11:28 [WARNING] Failed setting organization when creating original user: Not allowed to update an org without any users in the organization. Add at least one user to update
2024/10/01 23:11:28 [INFO] Starting cloud schedules for orgs if enabled!
2024/10/01 23:11:28 [DEBUG] Starting schedule setup for execution cleanup every 300 seconds. Running first immediately. Environments: []string{"Shuffle"}
2024/10/01 23:11:28 [INFO] Running schedule for cleaning up or re-running unfinished workflows in 1 environments.
2024/10/01 23:11:28 [INFO] Getting and validating workflowapps. Got 0 with err <nil>
2024/10/01 23:11:28 [DEBUG] Downloading default apps as 0 were found
2024/10/01 23:11:28 [DEBUG] Getting apps from url 'https://github.com/shuffle/python-apps'
2024/10/01 23:11:28 [WARNING] Api authentication failed in ABORT dangling executions: Couldn't find the user
2024/10/01 23:11:28 [DEBUG] Successfully ran workflow cleanup request for Shuffle. Body: {"success": false}
2024/10/01 23:11:28 [WARNING] Api authentication failed in Rerun executions: Couldn't find the user
2024/10/01 23:11:28 [DEBUG] Successfully ran workflow RERUN request for Shuffle. Body: {"success": false}
2024/10/01 23:11:32 Skipping auth setup: get_excel_file_data
2024/10/01 23:11:33 [WARNING] Skipping build of Yara to later
2024/10/01 23:11:33 [INFO] Starting build of 143 containers (FIRST)
2024/10/01 23:11:33 [INFO] Setting up memory build structure for folder: email/1.1.0/
2024/10/01 23:11:33 [INFO] Building email/1.1.0/ with proxy "". Tags: "frikky/shuffle:email_1.1.0". This may take up to a few minutes.
...

Oh, cheers! I think this may have to do with the fact this was built a long time ago when everything was User dependant instead of Org dependant as it is now.

Should be a quickfix. Assigning @yashsinghcodes to it

frikky avatar Oct 02 '24 10:10 frikky

haveing the exact same issue on kubernetes deployment we have configured HUFFLE_DEFAULT_USERNAME, SHUFFLE_DEFAULT_PASSWORD, and SHUFFLE_DEFAULT_APIKEY in all-in-one.yaml. When deploying we receive the same erros like mentioned from thread starter.

2024/11/08 12:53:52 [DEBUG] Inside Files Init with org bucket name "shuffle_org_files" 2024/11/08 12:53:52 [DEBUG] Starting Shuffle backend - initializing database connection 2024/11/08 12:53:52 [DEBUG] Starting with memcached address '' (SHUFFLE_MEMCACHED). If this is empty, fallback to default (appengine / local). Name: 'onprem' 2024/11/08 12:53:52 [DEBUG] Using custom opensearch url 'https://opensearch:9200' 2024/11/08 12:53:52 [DEBUG] Initialized Shuffle database connection. Setting up environment. 2024/11/08 12:54:02 [DEBUG] Starting INIT setup for Elasticsearch/Opensearch 2024/11/08 12:54:02 [DEBUG] Getting organizations for Elasticsearch/Opensearch 2024/11/08 12:54:02 [DEBUG] Running on backend-854f54777b-w9rrw:5001 2024/11/08 12:54:02 [DEBUG] Got 0 organizations to look into. If this is 0, we wait 10 more seconds until DB is ready and try again. 2024/11/08 12:54:12 [DEBUG] No orgs. Setting NEW org "default" 2024/11/08 12:54:12 [INFO] Cloud action executed successfully for 'init' 2024/11/08 12:54:12 [INFO] Ran initial setup! 2024/11/08 12:54:12 [INFO] Waiting 30 seconds during init to make sure the opensearch instance is up and running with security features enabled 2024/11/08 12:54:42 [INFO] Setting up 0 schedule(s) 2024/11/08 12:54:42 [INFO] Trying to set up user based on environments SHUFFLE_DEFAULT_USERNAME & SHUFFLE_DEFAULT_PASSWORD 2024/11/08 12:54:42 [DEBUG] Creating org for default user 2djklasdjlasdjlasjdjlkasdj 2024/11/08 12:54:42 [WARNING] Failed setting organization when creating original user: Not allowed to update an org without any users in the organization. Add at least one user to update 2024/11/08 12:54:42 [INFO] Starting cloud schedules for orgs if enabled! 2024/11/08 12:54:42 [DEBUG] Starting schedule setup for execution cleanup every 300 seconds. Running first immediately. Environments: []string{"test"} 2024/11/08 12:54:42 [INFO] Running schedule for cleaning up or re-running unfinished workflows in 1 environments. 2024/11/08 12:54:42 [WARNING] Api authentication failed in ABORT dangling executions: Couldn't find the user 2024/11/08 12:54:42 [DEBUG] Successfully ran workflow cleanup request for test. Body: {"success": false} 2024/11/08 12:54:42 [WARNING] Api authentication failed in Rerun executions: Couldn't find the user 2024/11/08 12:54:42 [DEBUG] Successfully ran workflow RERUN request for test. Body: {"success": false} 2024/11/08 12:54:43 [INFO] Getting and validating workflowapps. Got 46 with err 2024/11/08 12:54:43 [DEBUG] Skipping download of default apps as 46 were found 2024/11/08 12:54:43 [INFO] Downloading OpenAPI data for search - EXTRA APPS 2024/11/08 12:54:45 [INFO] Skipping download of extra API samples as 46 were found 2024/11/08 12:54:45 [INFO] Starting healthcheck job every 30 minute. Stats available on /api/v1/health/stats. Disable with SHUFFLE_HEALTHCHECK_DISABLED=true 2024/11/08 12:54:45 [DEBUG] Successfully started healthcheck interval of 30 minutes 2024/11/08 12:54:45 [INFO] Finished INIT (ES) 2024/11/08 12:54:45 [DEBUG] Ops dashboard api key or org not set. Getting first org and user that is valid 2024/11/08 12:54:45 [ERROR] Failed getting first org: Bad statuscode: 404

ccMatze avatar Nov 08 '24 12:11 ccMatze

@ccMatze Thank you for bringing it to our attention. I will keeping this in priority and will ship the fix ASAP.

yashsinghcodes avatar Nov 08 '24 13:11 yashsinghcodes